-- Logs begin at Tue 2022-06-14 23:36:21 UTC, end at Wed 2022-06-15 00:55:43 UTC. -- Jun 15 00:20:27.207141 dv-cinder-835384-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 00:20:27.221226 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 00:20:27 2022] *** Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: nodename: dv-cinder-835384-6-iscsi Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: machine: x86_64 Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: clock source: unix Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: pcre jit disabled Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: detected number of CPU cores: 4 Jun 15 00:20:27.223280 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: current working directory: / Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: detected binary path: /usr/bin/uwsgi-core Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: your processes number limit is 47680 Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: your memory page size is 4096 bytes Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: detected max file descriptor number: 2048 Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: lock engine: pthread robust mutexes Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: thunder lock: enabled Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 00:20:27.223844 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 00:20:27.267806 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: Python main interpreter initialized at 0x565287eac9c0 Jun 15 00:20:27.267806 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: python threads support enabled Jun 15 00:20:27.267806 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: your server socket listen backlog is limited to 100 connections Jun 15 00:20:27.267806 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: your mercy for graceful operations on workers is 90 seconds Jun 15 00:20:27.268126 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 00:20:27.268208 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: *** Operational MODE: preforking *** Jun 15 00:20:27.268491 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: *** uWSGI is running in multiple interpreter mode *** Jun 15 00:20:27.268491 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: spawned uWSGI master process (pid: 80256) Jun 15 00:20:27.268664 dv-cinder-835384-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 00:20:27.270168 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: spawned uWSGI worker 1 (pid: 80266, cores: 1) Jun 15 00:20:27.270168 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: spawned uWSGI worker 2 (pid: 80267, cores: 1) Jun 15 00:20:27.270168 dv-cinder-835384-6-iscsi devstack@keystone.service[80256]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 00:20:32.785699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 00:20:32.785699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: from cryptography.utils import int_from_bytes Jun 15 00:20:32.786441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 00:20:32.786441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: from cryptography.utils import int_from_bytes Jun 15 00:20:32.787505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 00:20:32.787505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: from cryptography.utils import int_from_bytes Jun 15 00:20:32.787961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 00:20:32.787961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: from cryptography.utils import int_from_bytes Jun 15 00:20:32.836084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 00:20:32.836275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 00:20:32.836471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] command line args: [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 00:20:32.836647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 00:20:32.836728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 00:20:32.836839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 00:20:32.836924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ================================================================================ {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 00:20:32.837029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] command line args: [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 00:20:32.837150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] admin_token = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.837295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 00:20:32.837399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] config_dir = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.837485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ================================================================================ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 00:20:32.837588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.837706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] admin_token = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.837788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] config_source = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.837936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] debug = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_dir = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_source = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] debug = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.838910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_config_append = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.839902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_dir = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_config_append = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_file = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_dir = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.840932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.841974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.842859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.843779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] publish_errors = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] publish_errors = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.844914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] standard_threads = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845870 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] standard_threads = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.845994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] use_journal = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] use_json = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_journal = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_json = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] use_stderr = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_stderr = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.846960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] use_syslog = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.847035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_syslog = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.847111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.847231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 00:20:32.847309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.847482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.847561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.847717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.847795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.847975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.848935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.849969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.850945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.851904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.852876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.853957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.854941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.855957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.856123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.856207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.856878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.856958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.857862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.858950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.859951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.860940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.861917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.862869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.863936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.864851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.865940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.866984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.867910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.868870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.869946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.870959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.871897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.872644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.957555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.957857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.958123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.external = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.958388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.958722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.958851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.password = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.external = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] auth.token = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.959867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.password = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.token = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.960874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.961881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.962884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.963872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.964862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.965963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.966954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.967931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.968913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.969869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.970995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.971945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.972994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.973972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.974960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.975977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.976979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.977920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.978979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.979957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.980905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.981057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.981135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.981813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.981897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.982871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.983981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.984988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.985929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.986902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.987853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.988975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.989924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.990975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.991994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.992861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] role.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] role.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.993934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.994942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.995946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.996957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.997594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.997594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.997842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.997973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.998984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:32.999997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.000861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.001989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.002883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.003905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.004937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.005965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.006993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.connection = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.007950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.008962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.009947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.010985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.011894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.012019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.012019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80266) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 00:20:33.012264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.012264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.012264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 00:20:33.012264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 00:20:33.039780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.039970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.040106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.040241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.040241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.040478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.048937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.049126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.076975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.077204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.077581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.077819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.130927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.131420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.179360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.179589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.180534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.180773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.245893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.246550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.268863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.269916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.270561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.281593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.281952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.294698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.294889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.295923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.296059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.296186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.296311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.296311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80266) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 00:20:33.312717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.312926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 00:20:33.313258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.313385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.313523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 00:20:33.313651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 00:20:33.313804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.313946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 00:20:33.314198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.314966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.315231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.325419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.325687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.325951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.326085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.326227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.326367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.326636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.326950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.334383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.335217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.337686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.337958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.338233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.338508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.338589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.338802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.338887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.339923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.340107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.340196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.340365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.340446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.341158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.341436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.341702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.367980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.369030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.369582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.369856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.370660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.370973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.375131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.375403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.375673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.376295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.376576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.376862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.383053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.383330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.384331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.384611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.387830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.388111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.388387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.388661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.388929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.389169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.389251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.389449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.389530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.389710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.389800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.390001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.390278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.390554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.390854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.391127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.408864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.409137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.410486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.410763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.413665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.415388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.417322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.417611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.419619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.419913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.423347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.425112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.427031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.428852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.430634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.430951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.431231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.431507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.431778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.432050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.432522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.432822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.433105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.433387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.433663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.433941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.452368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.452647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.452909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.453187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.454840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.455126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.455394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.455670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.461083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.462794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.463075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.466160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.466767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.467076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.467354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.467641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.467913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.467994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.468192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.471428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.471713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.471996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.472290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.472570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.482103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.482490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.482810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.483103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.483379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.483655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.483923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.486753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.487062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.487368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.487650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.487930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.488209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.488492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.501885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.505162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.506820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.508465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.510075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.510160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.511870 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.512142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.512416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.513976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.515636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.517431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.517711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.517991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.520206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.521822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.525049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.525335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.525418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.525614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.525883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.526985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.530818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.531112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.531397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.531680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.535749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.536033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.536308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.541157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.541446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.541727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.544172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.544448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.544769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.544996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.545271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.545550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.545819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.546096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.546461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.546733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.547026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.547291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.547552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.547807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.549694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80266) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 00:20:33.549974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.550255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.550529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.550828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.551110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.551385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.551669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.552042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.552314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.552586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.552857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.553123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.553384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80266) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 00:20:33.585634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x565287eac9c0 pid: 80267 (default app) Jun 15 00:20:33.589046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d400b-2b06-4057-9172-daffeb38f446 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:33.589345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d400b-2b06-4057-9172-daffeb38f446 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:33.589617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d400b-2b06-4057-9172-daffeb38f446 None None] PATH_INFO: `/v3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:33.592283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x565287eac9c0 pid: 80266 (default app) Jun 15 00:20:33.594437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Wed Jun 15 00:20:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:41.325001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:41.325572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:41.325572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:41.329881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:20:41.335109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:41.335487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:41.335658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:41.405267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: INFO dbcounter [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] Registered counter for database keystone Jun 15 00:20:41.426675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG oslo_db.sqlalchemy.engines [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80267) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 00:20:41.432852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writer thread running {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 00:20:41.608169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG passlib.handlers.bcrypt [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80267) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 00:20:41.608515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG passlib.handlers.bcrypt [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80267) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 00:20:41.661965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:41.663594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:41.786310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 00:20:41.948680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:42.021001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: INFO dbcounter [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] Registered counter for database keystone Jun 15 00:20:42.040526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG oslo_db.sqlalchemy.engines [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80266) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 00:20:42.046594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writer thread running {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 00:20:42.164455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72412a3-dc72-49f5-b65b-6686ca78165b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:42.168703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:42.169351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:42.169630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:42.169902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:42.177920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:42.230052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING keystone.common.rbac_enforcer.enforcer [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 00:20:42.230388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56138bcb-9235-4559-8823-79c59b43ac2e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:42.239798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Wed Jun 15 00:20:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:20:42.245627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:42.260358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271223-8fa0-4080-822e-7bf57049b2b6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:42.265298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:42.266396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:42.266674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:42.266970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] PATH_INFO: `/v3/projects/bc3f10f7de5140a991c9148b14dd9e73` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:42.268455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] RBAC: Authorizing `identity:get_project(project_id=bc3f10f7de5140a991c9148b14dd9e73)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:42.318149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 00:20:42.318486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d60fd-81d3-448d-ad12-20ea004d23f5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:42.323096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Wed Jun 15 00:20:42 2022] GET /identity/v3/projects/bc3f10f7de5140a991c9148b14dd9e73 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:44.405214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb357-4b5b-4eb5-bcf6-c5c51def4612 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:44.405744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb357-4b5b-4eb5-bcf6-c5c51def4612 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:44.405744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb357-4b5b-4eb5-bcf6-c5c51def4612 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:44.409950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:20:44.415628 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:44.416017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:44.416181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:44.435095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:44.436660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:44.444557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 15 00:20:44.468038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb357-4b5b-4eb5-bcf6-c5c51def4612 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:44.480622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb357-4b5b-4eb5-bcf6-c5c51def4612 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:44.485410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:44.486040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:44.486325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:44.486604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:44.494565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:44.496373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d560-50b5-449d-b08c-8a8038013ac5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:44.505138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Wed Jun 15 00:20:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:20:44.510520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:44.519475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6341e4a-b2ed-4027-8b6f-ada916a5528f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:44.523817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:44.524679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:44.524963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:44.525238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:44.526633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:44.528995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e628-bf63-436c-a148-d6362cb5ccb3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:44.530554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Wed Jun 15 00:20:44 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:46.720939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960241-4825-4fa4-ad8f-13a65c58e35d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:46.721474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7102975-5946-45c2-9e79-05cf4ee29cb2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:46.721474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7102975-5946-45c2-9e79-05cf4ee29cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:46.721909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960241-4825-4fa4-ad8f-13a65c58e35d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:46.721997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7102975-5946-45c2-9e79-05cf4ee29cb2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:46.722081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960241-4825-4fa4-ad8f-13a65c58e35d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:46.722855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 5/11] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:46.722999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:20:46.728140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:46.728267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70fe-6991-4955-a570-13614221c204 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:46.728430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:46.728521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70fe-6991-4955-a570-13614221c204 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:46.728712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:46.728806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70fe-6991-4955-a570-13614221c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:46.748763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:46.750075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:46.770693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG passlib.handlers.bcrypt [None req-9fad70fe-6991-4955-a570-13614221c204 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80266) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 00:20:46.771160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG passlib.handlers.bcrypt [None req-9fad70fe-6991-4955-a570-13614221c204 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80266) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 00:20:46.816345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9fad70fe-6991-4955-a570-13614221c204 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:46.817928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9fad70fe-6991-4955-a570-13614221c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:46.828643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 7/12] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:46.851193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:46.874747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:46.915683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 6/13] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:20:46.937984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad70fe-6991-4955-a570-13614221c204 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:46.954961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbbf56-e44b-42da-b83d-22a17e2b1641 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:46.988927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:46.989709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:46.989989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:46.990253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.001138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:20:47.012744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:20:47.013830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:20:47.014881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 8/14] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:20:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:20:47.021351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.038892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad70fe-6991-4955-a570-13614221c204 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.041314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.079757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.081233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.081636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.081993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.102582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.121252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b972fd0-ca78-4f24-9869-0de974770224 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.148732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.150565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.154976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.155626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.155905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.156176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.156918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:47.157222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.158597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.170360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 9/15] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:47.175530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.247405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 7/16] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:20:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 00:20:47.252939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.255573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.273214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317f49-2ccc-4ba5-ba11-a6939ceabb73 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.307680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.308541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.308851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.309121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.310309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.311781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.313366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 10/17] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:47.318546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.332398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.350624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804967-2142-4527-b650-2f2497dc7c83 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.383814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.384980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.384980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.385187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.403391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.404842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.417070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d284aeb-d8e9-4551-ba19-942869d3bfb8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:20:47.425190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:20:47.425974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:20:47.426770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:20:47.426770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:20:47.426770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:20:47.426770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:20:47.426770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 8/18] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:20:47.432025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.451177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.451968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.452244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.452511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.453686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.455140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.456855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 11/19] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:47.462233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:47.489215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.535464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb33f-8f6a-4bf1-80dc-eb4634ac5e04 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.544092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:47.563768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d992624-04ee-4828-a7bc-ec27903effc8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:47.569066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.569721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.570004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.570280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.571050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:47.571358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.572759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2811b-6fba-4174-b4b6-d9ab1d91ae34 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.590831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 9/20] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:47.597535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:47.598206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:47.598486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:47.598757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:47.599643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.domain.id=default, user.id=ba726016075244549882bd6014777a2e) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:47.599955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:47.601436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ed461-ddfe-4b8e-8378-159b266f4987 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:47.609748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 12/21] 10.222.0.33 () {56 vars in 1322 bytes} [Wed Jun 15 00:20:47 2022] GET /identity/v3/role_assignments?user.id=ba726016075244549882bd6014777a2e&scope.domain.id=default&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:50.024929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946e8c-caed-498a-b482-8e44a622b6ee None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.025820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946e8c-caed-498a-b482-8e44a622b6ee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.026329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946e8c-caed-498a-b482-8e44a622b6ee None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.028766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 10/22] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:50.036497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ebe5-1225-4514-b2ca-2e089d80fd79 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.041405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ebe5-1225-4514-b2ca-2e089d80fd79 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.041832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ebe5-1225-4514-b2ca-2e089d80fd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.072216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d008ebe5-1225-4514-b2ca-2e089d80fd79 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:50.073692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d008ebe5-1225-4514-b2ca-2e089d80fd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:50.181273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 13/23] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:50.247841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946e8c-caed-498a-b482-8e44a622b6ee None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:50.282864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadce7d-c455-4ba9-9a11-34f0aef8a6e5 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.283283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadce7d-c455-4ba9-9a11-34f0aef8a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.283769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadce7d-c455-4ba9-9a11-34f0aef8a6e5 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.285653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 14/24] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:20:50.305543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8cb2-a14d-4a94-b766-f631d6904d0b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.306254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8cb2-a14d-4a94-b766-f631d6904d0b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.306790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8cb2-a14d-4a94-b766-f631d6904d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.347892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-95fc8cb2-a14d-4a94-b766-f631d6904d0b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:50.349471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-95fc8cb2-a14d-4a94-b766-f631d6904d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:50.479681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946e8c-caed-498a-b482-8e44a622b6ee None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:50.505167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 15/25] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:20:50.506516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7d34-d502-4a6d-b57f-66f48f202857 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.511746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7d34-d502-4a6d-b57f-66f48f202857 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.512060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd7d34-d502-4a6d-b57f-66f48f202857 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.513385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 16/26] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:50.527139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:50.539329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.539537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.539974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.547456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:50.556890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:50.557662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.557944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.558287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.561467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:50.564152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:50.582401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:50.584172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:20:50.620723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:20:50.622291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:20:50.623186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:20:50.623186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:20:50.623186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:20:50.623186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:20:50.625180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 11/27] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:20:50.626078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:50.669738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 17/28] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:50.670811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:50.791817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e018f-2140-426e-9f15-11162f1ff81b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:50.821749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c0908-36a8-4284-ab8c-bec7e5fc9661 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:50.866961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:50.868131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.869769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.870227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.872050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:50.873358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:50.875626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:50.879085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:50.896518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:50.897193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:50.897473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:50.897747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:50.898502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:50.899915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd77b90-c073-4462-a4bb-90ddb8f2e4db None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:50.912752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 18/29] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:50.914121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.028965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea289ecf-ae72-4b03-8268-9b1132dfebcc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.043419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 12/30] 10.222.0.33 () {60 vars in 1131 bytes} [Wed Jun 15 00:20:50 2022] POST /identity/v3/roles => generated 219 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:20:51.045621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127a11-2b22-4000-b8b4-075ab1b1c44d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.046104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127a11-2b22-4000-b8b4-075ab1b1c44d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.046583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127a11-2b22-4000-b8b4-075ab1b1c44d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.050916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 13/31] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:51.053879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127a11-2b22-4000-b8b4-075ab1b1c44d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.080251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.083057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.085070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.086427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.109234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:20:51.109419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9bccb8e2-675a-4563-9205-a5e6d5c28afd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:20:51.126243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:20:51.127222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:51.127974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:20:51.136035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 19/32] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:20:51.137742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.138180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.138736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.172442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:51.176250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:51.247609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127a11-2b22-4000-b8b4-075ab1b1c44d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.250886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.282804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 20/33] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:51.282804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.294672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.295704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.296150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.296657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.297833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:51.299664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:51.347028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 14/34] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/projects => generated 301 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:20:51.348207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.441788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daccab6-bec3-495f-a700-32afe01fc1cb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.508377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.509086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.509367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.509681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.510432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:51.510762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:51.512255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:51.527201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 21/35] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:51.533073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.595297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.607227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0f90-dcfa-47ca-9a73-770f8dd595be None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.638769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.648536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.649235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.649648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.649929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.650705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:51.652252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090959a-f449-477f-bbf9-89be2505ee0a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:51.671678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 15/36] 10.222.0.33 () {60 vars in 1131 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/roles => generated 225 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:20:51.673550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f51c3-04f3-462c-a717-88b162847a00 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.673852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f51c3-04f3-462c-a717-88b162847a00 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.674202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f51c3-04f3-462c-a717-88b162847a00 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.675645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 16/37] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:51.685246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec10df-bae3-47af-8683-6e287aba6ff4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.685580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec10df-bae3-47af-8683-6e287aba6ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.685814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec10df-bae3-47af-8683-6e287aba6ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.707509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-40ec10df-bae3-47af-8683-6e287aba6ff4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:51.709166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-40ec10df-bae3-47af-8683-6e287aba6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:51.721507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12fc48-19ce-4909-b2ba-c6545fdf47e3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.767872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.768769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.769054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.769330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.771388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:51.773557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fd2c2-ee29-498c-892c-f757062b7dc8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:51.775915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 22/38] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:20:51 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:51.777602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc055a-2dcf-4943-a7e0-a800888d82a8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.777892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc055a-2dcf-4943-a7e0-a800888d82a8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.778170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc055a-2dcf-4943-a7e0-a800888d82a8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.779768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 23/39] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:51.783790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc055a-2dcf-4943-a7e0-a800888d82a8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.807878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 17/40] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:20:51.809732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.810023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.810300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.828281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:51.829512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:51.898366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.904543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 18/41] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:51.905392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:51.926387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc055a-2dcf-4943-a7e0-a800888d82a8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:51.941703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:51.961950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:51.962737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:51.963046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:51.963320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:51.964488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:51.966473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:51.968109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 24/42] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:51.968913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:52.015963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1af13-25f5-42bf-b1b2-2ecaa0bf3e3b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:52.050581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:52.051942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:52.052274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:52.052552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:52.053838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:52.055372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:52.057261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 19/43] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:52.058025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:52.132046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:52.167565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed253-55e2-4048-a54b-5a35dbd49e87 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:52.178956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2022-31a3-4577-abbb-3fc280c4649a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:52.201141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:52.202010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:52.202287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:52.202573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] PATH_INFO: `/v3/domains/default/users/ba726016075244549882bd6014777a2e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:52.204840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ba726016075244549882bd6014777a2e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:52.206286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:52.213624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:52.214299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:52.214613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:52.215531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:52.216504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:52.218045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415fe83-bef8-4012-a6f8-bc42fe95ce31 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:52.230951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 20/44] 10.222.0.33 () {58 vars in 1361 bytes} [Wed Jun 15 00:20:52 2022] PUT /identity/v3/domains/default/users/ba726016075244549882bd6014777a2e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:20:52.230951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:52.332910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b75-5ca8-4eb8-a7e7-a12c8f360c30 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:52.349594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 25/45] 10.222.0.33 () {60 vars in 1131 bytes} [Wed Jun 15 00:20:51 2022] POST /identity/v3/roles => generated 223 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:20:52.364396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:52.375426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:52.376566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:52.377286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:52.379503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:52.379503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:52.380041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c5b58-14db-48dd-8ede-daff375d137c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:52.411796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 21/46] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:20:52 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:20:54.340988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee6a1-f1ed-4b43-995e-8daccfa49e8d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.341510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee6a1-f1ed-4b43-995e-8daccfa49e8d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.341617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ee6a1-f1ed-4b43-995e-8daccfa49e8d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.342921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 26/47] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:54.347957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef46ae-bcb4-41db-8088-1aa6721443e6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.348372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef46ae-bcb4-41db-8088-1aa6721443e6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.348516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef46ae-bcb4-41db-8088-1aa6721443e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.365046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-48ef46ae-bcb4-41db-8088-1aa6721443e6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:54.366235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-48ef46ae-bcb4-41db-8088-1aa6721443e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:54.434408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 22/48] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:54.457269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ee6a1-f1ed-4b43-995e-8daccfa49e8d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.508809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:54.545801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7176cac-8fcb-43d4-8860-a0a251b924f9 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.546091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7176cac-8fcb-43d4-8860-a0a251b924f9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.546375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7176cac-8fcb-43d4-8860-a0a251b924f9 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.547803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 23/49] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:20:54.552689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.552985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.553265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.567558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ee6a1-f1ed-4b43-995e-8daccfa49e8d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:54.569891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:54.571103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:54.600708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:54.601427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.601708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.601980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.612638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:20:54.620275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:20:54.621847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:20:54.622801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 27/50] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:20:54.627682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.640116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 24/51] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:54.662437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.702421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:54.727187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a632c76-6be2-4358-b46d-4ae00bfaa5f2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:54.728705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:54.760415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5a812-5431-4b8d-abf6-939f05a14239 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:54.761875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:54.762543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.762855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.763139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.763892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:54.764210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:54.765728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:54.773769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 28/52] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:54.779280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.793405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:54.794195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.794482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.794761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.795970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:54.797438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:54.799406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 25/53] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:54.805021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.879320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5f816-c87d-4a15-b200-3bceb57c712a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:54.903568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4bf6-19a9-4d34-96d3-83acf880b49d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:54.911736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:54.912602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.912884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.913161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.913252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:54.914468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:54.915951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:54.917595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 29/54] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:20:54 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:54.923429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:54.939640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:54.940290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:54.940571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:54.940848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:54.941552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:54.943030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:54.953237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:54.968133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 26/55] 10.222.0.33 () {60 vars in 1140 bytes} [Wed Jun 15 00:20:54 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:20:55.026189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc63fd-a003-4395-bbdf-58cbd09df857 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:55.062317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:55.063155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:55.063435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:55.063706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:55.065013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:55.066481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f41899-d307-48fe-93b0-facad4252a7b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:55.068173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 30/56] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:20:55.074138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:55.176086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b84b-cfbf-4be0-84c0-3ca5b0bdb83f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:55.207813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:55.208472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:55.208754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:55.209029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:55.209862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.domain.id=default, user.id=ba726016075244549882bd6014777a2e) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:20:55.210176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:55.211683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3e72-7c53-445d-8071-6f0802ce1f5c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:55.220632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 27/57] 10.222.0.33 () {56 vars in 1322 bytes} [Wed Jun 15 00:20:55 2022] GET /identity/v3/role_assignments?user.id=ba726016075244549882bd6014777a2e&scope.domain.id=default&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:57.076677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef057db-1ded-495d-945d-130bfa3a1109 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:57.077195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef057db-1ded-495d-945d-130bfa3a1109 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:57.077306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef057db-1ded-495d-945d-130bfa3a1109 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:57.078587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 31/58] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:20:57.083860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:57.084271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:57.084424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:57.101332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:20:57.102520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:20:57.119444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:57.168173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 28/59] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:20:57.191951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef057db-1ded-495d-945d-130bfa3a1109 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:57.285954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:20:57.289647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef057db-1ded-495d-945d-130bfa3a1109 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:57.322224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:57.323039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:57.323326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:57.323603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:57.325391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:57.326921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75fae2-8ed2-455a-93d0-5b51676decd2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:57.328464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 32/60] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:20:57.333821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:20:57.428614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b78573-d362-444f-a3a1-99ad5c94c3a9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:20:57.461521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:20:57.462173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:20:57.462459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:20:57.462737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:20:57.463481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:20:57.464960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc492c-f516-445c-acd4-fd16058ad26b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:20:57.489474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 29/61] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:20:57 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:00.698312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea353bd9-d0df-49e0-bfe9-f3c0ecca8fee None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:00.699726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea353bd9-d0df-49e0-bfe9-f3c0ecca8fee None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:00.700139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea353bd9-d0df-49e0-bfe9-f3c0ecca8fee None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:00.701681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 33/62] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:00.707161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47866ede-a59f-4f3d-9b6b-ebdc11226600 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:00.707915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47866ede-a59f-4f3d-9b6b-ebdc11226600 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:00.708360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47866ede-a59f-4f3d-9b6b-ebdc11226600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:00.735834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-47866ede-a59f-4f3d-9b6b-ebdc11226600 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:00.737335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-47866ede-a59f-4f3d-9b6b-ebdc11226600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:00.740603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:00.833200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 30/63] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:00.854926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898536-9202-4cfe-83da-8d45aee87c56 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:00.855520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898536-9202-4cfe-83da-8d45aee87c56 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:00.855990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898536-9202-4cfe-83da-8d45aee87c56 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:00.857548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 34/64] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:00.862645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898536-9202-4cfe-83da-8d45aee87c56 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:00.863028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:00.863370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:00.863661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:00.897215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:00.898423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:01.026507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 31/65] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:01.067533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898536-9202-4cfe-83da-8d45aee87c56 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.095133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.115274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.116080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.116367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.116645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.128245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ddbffc45-ad7a-4d6f-abbd-c6a0fbdf1b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:01.135965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:01.137432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:01.138191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 35/66] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:01.139359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d70-08cc-4d9d-beb9-82f47b3d0c84 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.139359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d70-08cc-4d9d-beb9-82f47b3d0c84 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.139645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d70-08cc-4d9d-beb9-82f47b3d0c84 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.142814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 36/67] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:01.150820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86959d70-08cc-4d9d-beb9-82f47b3d0c84 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.192169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.193477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.238474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e69b8-a824-47c1-b075-6eaf738bf706 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.273463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.274278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.274559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.274862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.286123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:01.290448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86959d70-08cc-4d9d-beb9-82f47b3d0c84 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-d7e1df57-fd70-4690-9b1e-a5c44e446f00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:01.296021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:01.296881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:01.297640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 32/68] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:01.299225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.299225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.299225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.315935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:01.317117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:01.325646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.326315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.326601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.326914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.327671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:01.327982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.329371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308cbc5-a45b-4513-b7e8-ce7049f9dfa8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:01.337729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 37/69] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:01.338960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7949d4-09ab-438b-9c1a-8e7ba38529f6 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.339285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7949d4-09ab-438b-9c1a-8e7ba38529f6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.339568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7949d4-09ab-438b-9c1a-8e7ba38529f6 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.340947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 38/70] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:01.341875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2d71-ca51-47ca-a45a-997e9c296115 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.342231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2d71-ca51-47ca-a45a-997e9c296115 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.342514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2d71-ca51-47ca-a45a-997e9c296115 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.343762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 39/71] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:01.344461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2d71-ca51-47ca-a45a-997e9c296115 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.389272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 33/72] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:01.390471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.436482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.447341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2d71-ca51-47ca-a45a-997e9c296115 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.449417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.481342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.482007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.482283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.482552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.483328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:01.483641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.485019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122482-d31d-46a8-b6ae-4c116073a843 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:01.489794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda911c-74e1-497a-adba-4c31338fe340 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.493967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 40/73] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:01.495279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.496143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.496420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.514517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:01.516720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:01.523844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.524718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.525002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.525280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.526491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.528113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e55ce-3e63-4926-bbd5-4208f58903a4 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:01.534125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 34/74] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:01.534125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.534125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.534125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.547797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:01.549000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:01.590180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 41/75] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:01.591058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.617422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 35/76] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:01.618275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.681674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.690083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.699769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f72d7d-45d8-4550-aa5b-bc78b4fb3dee None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.732190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833607c9-0335-46dd-a544-f7164d0255f0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.734389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.735231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.735517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.735792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.745706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:01.753851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:01.754963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:01.755916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 42/77] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:01.756624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.769574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.770458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.770729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.771034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.786135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.787628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:01.789288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 36/78] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:01.790047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.860264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5dec-b006-474b-91b8-d786021ede2c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.890019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596202-e9e6-45f7-a803-e912fd0a85ac None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:01.892473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.893125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.893413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.893692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.905808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.906207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:01.924385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:01.925172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:01.925462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:01.925740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:01.926946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.928442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:01.930393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 37/79] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:01.930941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:01.961501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:01.962940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.028569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beab2e-1983-49ae-85d3-1791df4f1e22 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.060567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 43/80] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:02.061347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.061464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.062201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.062482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.062753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.073197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:02.081304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:02.082311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:02.083225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 38/81] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:02.083984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.108046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.130799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.163826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f5b1e-403d-4623-8286-f57bf215c3b2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.184717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410bb32-8b7a-4d16-9a24-2309fec89e3d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.196945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.197637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.197916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.198187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.199521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:02.199826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.201197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.209040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 44/82] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.210019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.218819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.219412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.219700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.219982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.286599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.288093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.308702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2853-f5ad-4c09-a6ed-9bdc85d12950 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.310994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.332217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.342555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.343242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.343530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.343807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.344741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.345931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.371061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 45/83] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:02 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:02.371903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.392817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 39/84] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:02.393929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.505320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79072e4-3997-4dfc-bed2-87f38e86b027 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.508385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef781-9f54-4576-a94a-6c3c7761cd9d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.544822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.546048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.546048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.546230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.547770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.547886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.547957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.548036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.548254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.549393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:02.549490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.549558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.554984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.555142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 46/85] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.555142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.564795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 40/86] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.565612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.575547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.589503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.675757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c0d6b-03fb-4528-a926-1319ec668019 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.694759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06516c6-c63a-4fd3-9da6-933de401e393 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.718976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.720438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.720960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.721385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.722610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.724170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.725971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 47/87] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.726537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.739183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.740054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.740334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.740608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.741819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.743337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.745103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 41/88] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.745847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.829722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.844384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:02.851150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddc065-dca4-43b9-9935-820ed2fb638b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.858399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a6378-143c-40e5-a610-3da41deed442 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:02.889979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.891166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.891796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.892270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.893401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=ba726016075244549882bd6014777a2e) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:02.893598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:02.894087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.894471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:02.894750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:02.895076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:02.896276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:02.896618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.897771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:02.899437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 42/89] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.900256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:02.909809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 48/90] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/role_assignments?user.id=ba726016075244549882bd6014777a2e&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:02.911071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:03.009268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc3d6-0635-4aa8-a3e5-41eebb5e33e1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.051492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.052164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.052449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.052725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.079187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:03.106513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ecce2-8c5a-4d02-8084-e08305a32f6e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.158550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.160124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.176383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.177050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.177334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.177610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.178560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:03.178974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.180540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.187377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 49/91] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:03.188285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:03.234717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:03.317105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 43/92] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:03.322348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:03.385389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:03.416857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002899e-7542-43f9-acda-e17e1f59fca1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.458235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a6d-b5b8-4d1b-a234-e6ed5019749d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.492389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.493063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.493412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.493613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.497579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.498452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.498733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.499038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.500239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.501702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.504131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 44/93] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:03 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:03.511979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:03.649472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.650800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.672163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ebfba-8da9-487c-b006-376f16ba716e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.683740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:03.707587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:03.722800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.722800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.722800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.722800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.723154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:03.723154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.723154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.733082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 45/94] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:03 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=87cd2d147f344fa99848788e1a735d8f => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:03.798030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 50/95] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:03.803760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:03.942228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459442c-3243-4289-9cf2-f1972a1e1c75 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:03.983211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:03.984092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:03.984376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:03.984650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] PATH_INFO: `/v3/projects/7980010e74fe43cea131486b5e951608` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:03.985855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] RBAC: Authorizing `identity:get_project(project_id=7980010e74fe43cea131486b5e951608)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:03.987335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3607f-6bf5-450f-830c-45fa8f11ed7f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:03.989346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 46/96] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:03 2022] GET /identity/v3/projects/7980010e74fe43cea131486b5e951608 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:03.997949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:04.131650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1638bd8-1a5a-44e9-ad2d-e3573d8a0ca1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:04.157482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:04.174367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:04.175338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:04.175850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:04.176325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:04.177405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=7980010e74fe43cea131486b5e951608, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:04.177927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:04.179593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118f7c9-08f3-485f-a694-422e8149cc35 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:04.185772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 51/97] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:03 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=7980010e74fe43cea131486b5e951608&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:05.225628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0482f-f9ad-4ab1-9530-a39386c6f9fd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.226145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0482f-f9ad-4ab1-9530-a39386c6f9fd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.226417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0482f-f9ad-4ab1-9530-a39386c6f9fd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.227872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 47/98] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:05.238259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.238742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.238838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.259960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:05.261236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:05.310135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f23f-f5c9-436b-a15f-e69dc38961d3 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.310419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f23f-f5c9-436b-a15f-e69dc38961d3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.310799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78f23f-f5c9-436b-a15f-e69dc38961d3 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.312438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 48/99] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:05.317225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.317515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.317791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.338035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:05.344456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:05.352301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 52/100] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:05.389824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:05.415136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:05.447196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 49/101] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:05.480608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:05.568746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5fbbb-0c27-4094-9405-ea38cff66c2f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:05.599131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:05.628847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:05.629657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.629946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.630229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.635508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:05.637037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:05.638706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 53/102] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:05.644372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:05.644746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8359d-b350-43f6-8990-1af2658e48bd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:05.739578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:05.740321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.740607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.740883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.767739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:05.775509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:05.776363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:05.777770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:05.783207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 50/103] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:05.788775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:05.834082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:05.855101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea1672-e12b-4773-a190-7f7f0ea912c8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:05.901275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:05.902962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:05.903478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:05.903953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:05.904881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:05.906611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:05.943474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 54/104] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:21:05 2022] POST /identity/v3/projects => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:21:05.955621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cb5d8-add0-4213-b187-40db06f4068f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.013799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.013799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.014066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.014400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.015274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:06.015592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:06.017028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b4118-11f5-4ad0-873f-8a6e3ede40f5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:06.029275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 51/105] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:06.035119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.076270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b0d14-7a20-41bf-baf0-d3946b65bb4d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.076677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b0d14-7a20-41bf-baf0-d3946b65bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.076850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b0d14-7a20-41bf-baf0-d3946b65bb4d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.078546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 52/106] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:06.084049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.084366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.084611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.086583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.119068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:06.120953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:06.162311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a2c7f-1689-444e-9b29-38736c00a563 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.202054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.203381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.203881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.204351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] PATH_INFO: `/v3/users/8a66399190ef4fb898e50aa8a146b4a5` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.220563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] RBAC: Authorizing `identity:get_user(user_id=8a66399190ef4fb898e50aa8a146b4a5)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:06.222391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:06.224716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 55/107] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/users/8a66399190ef4fb898e50aa8a146b4a5 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:06.231315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.264251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 53/108] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:06.309372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.342046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb426b-c00d-45fa-b60e-2ded595d644c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.346265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.364092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.391145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.391858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.392523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.392805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.473576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:06.481826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:06.595817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.605241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 56/109] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:06.606911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd2e2-6ef3-4f1e-92f7-a6232dc09155 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.611694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.700122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.700946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.701229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.701504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.724735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:06.725710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f8527-3b3e-45cd-8a4a-6f10e575865f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-e44574c0-a81a-4cc9-af7d-505a7197af81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:06.735175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:06.736051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:06.736792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:06.738101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 54/110] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:06.740030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc63a6-cb11-4515-a758-d9e4b1a72d48 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.740416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc63a6-cb11-4515-a758-d9e4b1a72d48 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.740790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc63a6-cb11-4515-a758-d9e4b1a72d48 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.742924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 55/111] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:06.744103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc63a6-cb11-4515-a758-d9e4b1a72d48 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.769051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.769914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.770192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.770466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.771788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:06.773358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadfe43-c692-4f14-ae5d-8c07e7544118 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:06.775646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 57/112] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:06.777319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.777884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.778448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.797304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:06.798594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:06.856748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.876636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc63a6-cb11-4515-a758-d9e4b1a72d48 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:06.877512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:06.888154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 58/113] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:06.894590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:06.945332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:06.946006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:06.946286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:06.946561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:06.947354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:06.947663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:06.953652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:06.970918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 56/114] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:06.972907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.070598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a7bb6-d276-4cad-96ab-452496be45ea None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.126050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.127144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.127632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.128155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.129235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=8a66399190ef4fb898e50aa8a146b4a5) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:07.129781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.131545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.153127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 59/115] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/role_assignments?user.id=8a66399190ef4fb898e50aa8a146b4a5&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=205663cf9d3242d4a020abca619acbee => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.154057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.164397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c66f-cacc-439b-8fb7-415fb781640f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.219312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.220133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.220415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.220689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.248072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-62d7e51e-dcee-467a-82df-ddab4ba99f70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:07.255987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:07.256934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:07.257736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 57/116] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:07.259272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94351fa-45bb-43eb-b430-33c133622a6d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.259984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94351fa-45bb-43eb-b430-33c133622a6d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.261247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94351fa-45bb-43eb-b430-33c133622a6d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.263331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 58/117] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:07.273784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94351fa-45bb-43eb-b430-33c133622a6d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.284961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.289682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.368527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9609591-9b46-4a23-9a0a-b8de3b02d53b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.427142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.427947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.428228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.428500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.429771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.431250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e52f36-1e4d-4170-bc16-3f8ba069e71e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.433081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 60/118] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.434065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.434351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.434627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.453532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:07.454738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:07.461133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94351fa-45bb-43eb-b430-33c133622a6d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.508255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.510250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.511352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.511907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.513096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:07.513576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.515316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3561-dbc6-48fc-a2bf-bfdbab46abe2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.525773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 59/119] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.528056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148283c-092c-4853-8603-516dc5a694fa None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.528612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148283c-092c-4853-8603-516dc5a694fa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.529089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148283c-092c-4853-8603-516dc5a694fa None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.530624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 60/120] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:07.531583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148283c-092c-4853-8603-516dc5a694fa None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.544751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 61/121] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:07.550793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.617894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.633720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.679625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cf87f-d112-4a95-b331-cd9b508b27f9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.680005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148283c-092c-4853-8603-516dc5a694fa None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.719299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.720525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.721027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.721542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.723096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.724795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94be54-4de7-466b-826a-05a04e7b6749 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.726467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.727415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 62/122] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.729113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.730258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.730738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.732341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.732679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.732974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.734216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.735719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.737789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 61/123] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.743211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.751338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:07.752763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:07.825333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 63/124] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:07.826478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.870094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8539-f94d-48cb-9e08-b10d76be8c8d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.885724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.912644 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.913567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.913862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.914144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:07.928329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:07.935134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041699-6364-440f-9a8f-eb6524d203ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:07.936955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:07.937808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:07.938550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 62/125] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:07.939201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:07.986502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:07.987808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:07.988296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:07.988758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:07.990235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:07.992046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:07.994159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 64/126] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:07.995182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.053724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb087a6-a922-45ef-bdfc-9230ada5ab17 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.089985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.090919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.091210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.091562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/ba726016075244549882bd6014777a2e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.093588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, user_id=ba726016075244549882bd6014777a2e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.095175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.104215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5d05a-4457-4f2a-a34c-4376d53c2319 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.114404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 63/127] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:07 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/ba726016075244549882bd6014777a2e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:08.115591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.152367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.176726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.177535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.177819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.178092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.203502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:08.219265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:08.220206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 65/128] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:08.221476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.231670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.318225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51595b2a-83b5-4d01-9da2-224c1cf2e1e1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.374927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.376410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.376880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.377740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.379015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:08.379509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.381826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.394129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 64/129] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:08.395742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.406174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb5764-7211-4c26-86c5-20c935f2f312 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.443903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.444960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.445337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.445721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/d7a6b76f2705487dafd371ca25ccca47/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.449433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, user_id=d7a6b76f2705487dafd371ca25ccca47, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.451037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.471878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 66/130] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:08 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/d7a6b76f2705487dafd371ca25ccca47/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:08.473105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.527034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d86e-65d1-45c7-873c-b647af01a741 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.530428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.545910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.566811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.568033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.568386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.568731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.569570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:08.569942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.571715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.582454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 65/131] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:08.588218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.607661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1fc59-de82-4bb6-ba16-9297d73ba723 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.692077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.692968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.693250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.693527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.699508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.700982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.701769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0b6b-330f-4cbf-8102-3f80c461af0b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.702637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 67/132] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:08.708414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.755558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.756460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.756746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.757023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.758308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.759808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.761738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 66/133] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:08.766989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.821828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.906646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82f2f7-bd99-40ce-bd30-07bca21c4fb3 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.913076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696990-06e4-494b-a21c-bd1208a9ba85 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:08.925023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:08.955226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.956983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.957494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.957971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.959495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.961650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.964352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 68/134] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:08.970010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:08.987551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:08.988542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:08.988750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:08.988980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] PATH_INFO: `/v3/projects/7980010e74fe43cea131486b5e951608` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:08.990360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] RBAC: Authorizing `identity:get_project(project_id=7980010e74fe43cea131486b5e951608)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:08.991774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:08.993501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 67/135] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:08 2022] GET /identity/v3/projects/7980010e74fe43cea131486b5e951608 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:08.999575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:09.084635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c73-a372-4914-b3ab-dc6800bfc7ed None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:09.121853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:09.123157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.123686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.124162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/d7a6b76f2705487dafd371ca25ccca47/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.126946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, user_id=d7a6b76f2705487dafd371ca25ccca47, role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:09.128615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a277ee-7a8f-4e9b-b6c3-280d04429d99 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:09.158322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8ff1-7e51-488e-a0ab-9e499b8fa43a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:09.161012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 69/136] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:08 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/d7a6b76f2705487dafd371ca25ccca47/roles/87cd2d147f344fa99848788e1a735d8f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:21:09.187216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:09.200794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:09.201717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.202248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.202532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] PATH_INFO: `/v3/projects/7980010e74fe43cea131486b5e951608/users/d7a6b76f2705487dafd371ca25ccca47/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.204674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] RBAC: Authorizing `identity:create_grant(project_id=7980010e74fe43cea131486b5e951608, user_id=d7a6b76f2705487dafd371ca25ccca47, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:09.206305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4426a1-7cac-422b-aa39-cb768cd7d504 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:09.224699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:08 2022] PUT /identity/v3/projects/7980010e74fe43cea131486b5e951608/users/d7a6b76f2705487dafd371ca25ccca47/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:21:09.277482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a549e3-16d6-450d-84f5-43ccd8c32672 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.277790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a549e3-16d6-450d-84f5-43ccd8c32672 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.278049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a549e3-16d6-450d-84f5-43ccd8c32672 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.279512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 70/138] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:09.285459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6c0e-923d-4002-ac1c-81c22fa0a557 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.285898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6c0e-923d-4002-ac1c-81c22fa0a557 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.286189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6c0e-923d-4002-ac1c-81c22fa0a557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.312382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-737e6c0e-923d-4002-ac1c-81c22fa0a557 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:09.313694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-737e6c0e-923d-4002-ac1c-81c22fa0a557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:09.424287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 69/139] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:09.459131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a549e3-16d6-450d-84f5-43ccd8c32672 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:09.461171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4ae2-94ba-4f42-b444-c1da1488d2b3 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.461466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4ae2-94ba-4f42-b444-c1da1488d2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.461750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4ae2-94ba-4f42-b444-c1da1488d2b3 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.463229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 70/140] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:09.469233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.469529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.469811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.500820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:09.503041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:09.518737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:09.594177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 71/141] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:09.601132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a549e3-16d6-450d-84f5-43ccd8c32672 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:09.620189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:09.636738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:09.637547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.637835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.638117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.639431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:09.640892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:09.647907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 71/142] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:09.650480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:09.684342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:09.764704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7a2e-82c2-4a15-a9b6-095962d5013d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:09.775992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320459bf-cdcb-43b4-b902-f0b213e0d0da None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:09.779207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:09.806120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:09.806833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.807128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.807731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.808576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:09.810171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:09.836319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:09.837123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:09.837415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:09.837706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:09.847015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 72/143] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:09 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:21:09.853389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:09.861807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:09.862720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:09.863657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:09.871110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:10.017116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f91ec-9967-406d-8a23-23bd103c46f6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:10.059389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:10.060447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:10.060921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:10.061397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:10.062349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:10.062891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:10.064540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:10.079813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:10.085701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:10.217197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:10.223328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7773a-eb8a-42a0-bf87-60d810dc1272 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:10.272477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:10.273292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:10.273574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:10.273850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] PATH_INFO: `/v3/users/8a66399190ef4fb898e50aa8a146b4a5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:10.275125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] RBAC: Authorizing `identity:get_user(user_id=8a66399190ef4fb898e50aa8a146b4a5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:10.276640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:10.278575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:10 2022] GET /identity/v3/users/8a66399190ef4fb898e50aa8a146b4a5 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:10.284027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:10.311002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:10.391915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077063dd-9b28-40c5-865f-aa52a84641a8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:10.428595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:10.429464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:10.429793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:10.430070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:10.431346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:10.432887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adf893-f227-43aa-9366-e6e8b10c658c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:10.434647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:10 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:10.440369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:10.567027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a844d5f-b733-4b3d-a0b4-2879f99e9136 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:10.608751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:10.609690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:10.609975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:10.610246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/8a66399190ef4fb898e50aa8a146b4a5/roles/205663cf9d3242d4a020abca619acbee` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:10.612293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, user_id=8a66399190ef4fb898e50aa8a146b4a5, role_id=205663cf9d3242d4a020abca619acbee)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:10.613753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c00f8-616c-4869-beb4-fc8eab89f94d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:10.631471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:10 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/users/8a66399190ef4fb898e50aa8a146b4a5/roles/205663cf9d3242d4a020abca619acbee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:11.368987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db97b6-9db3-4668-af91-0f683aa41a8f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:11.369525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db97b6-9db3-4668-af91-0f683aa41a8f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:11.369525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db97b6-9db3-4668-af91-0f683aa41a8f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:11.371706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 75/149] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:11.378067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:11.378357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:11.378635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:11.399825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:11.401017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:11.425324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:11.501469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 75/150] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:11.555955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db97b6-9db3-4668-af91-0f683aa41a8f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:11.661759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:11.674680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db97b6-9db3-4668-af91-0f683aa41a8f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:11.711395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:11.715667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:11.715958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:11.716238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:11.727515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:11.740453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:11.741364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:11.742119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 76/151] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:11.749958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:11.879269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8129ff-cd4d-4d09-b108-f1ac25c57178 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:11.917680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:11.918270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:11.918558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:11.918856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:11.919617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:11.919925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:11.921331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:11.934938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 76/152] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:11.940696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:12.074685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161582-0550-46a3-985e-75ddb1a67c11 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:12.117810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:12.118686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.118988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.119260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] PATH_INFO: `/v3/users/ba726016075244549882bd6014777a2e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.123489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ba726016075244549882bd6014777a2e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:12.124953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:12.126754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 77/153] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:11 2022] GET /identity/v3/users/ba726016075244549882bd6014777a2e => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:12.132435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:12.166499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:12.278138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b867fa2-5a0a-4398-a4f4-371599e6af1a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:12.315881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:12.316536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.316820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.317096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.400816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:12.402868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:12.428159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:12.543604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 77/154] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:12.548972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:12.598238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:12.725741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441d4fc-d019-483c-97bf-b99007ba07b1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:12.765000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:12.765868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.766147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.766419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.767686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:12.769147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01173c17-dfe6-4c34-be07-b5dfe93e954b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:12.771048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 78/155] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:12 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:12.776866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:12.828362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e86ac-5700-4253-9663-05a0c72a94cd None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.828650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e86ac-5700-4253-9663-05a0c72a94cd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.828832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e86ac-5700-4253-9663-05a0c72a94cd None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.830206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 79/156] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:12.845127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ace0-22bf-49a7-8a11-cd526bcf48d6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.845425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ace0-22bf-49a7-8a11-cd526bcf48d6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.845677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ace0-22bf-49a7-8a11-cd526bcf48d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.867776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d791ace0-22bf-49a7-8a11-cd526bcf48d6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:12.868981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d791ace0-22bf-49a7-8a11-cd526bcf48d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:12.926483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27aa5-27ac-4ebf-9268-7390fcf57eac None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:12.959070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:12.989380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 80/157] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:12.992185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d912-a37f-4c83-8377-002e8a8dc5d1 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:12.992480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d912-a37f-4c83-8377-002e8a8dc5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:12.992741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d912-a37f-4c83-8377-002e8a8dc5d1 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:12.994164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 81/158] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:12.998798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:12.999775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.000257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.000674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.001666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=ba726016075244549882bd6014777a2e) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:13.002133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:13.003754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:13.005149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.007538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.007961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.021164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 78/159] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:12 2022] GET /identity/v3/role_assignments?user.id=ba726016075244549882bd6014777a2e&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:13.036662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:13.038088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:13.063152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:13.105873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.161297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 82/160] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:13.256633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.281654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769288d4-1541-4651-b0de-ea5a1dc8d81b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:13.330566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:13.331413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.331698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.331972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.343328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:13.356151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:13.357051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:13.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 79/161] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:13.363214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.429275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:13.431286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344b91e-496e-4779-8201-b9afa70fe4d6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:13.482892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:13.487717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.488228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.488657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.501426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:13.515013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:13.515904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:13.516721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:13.524492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 83/162] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:13.529514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7e3e4-cc32-4f56-bfe1-b1865a5cc3d8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:13.529864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.559134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:13.582191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:13.583453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.583882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.584304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.585240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:13.585695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:13.587275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:13.601235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 80/163] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:13.607764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.733719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3dd7d-2e6e-48f9-8f3d-6c9e6133f3cc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:13.735123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9611d-b5a8-44d1-83e7-8c5b37e2ad89 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:13.736859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:13.820232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:13.820931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.821211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.821487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.822263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:13.822576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:13.823595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:13.824072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:13.824497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:13.824775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:13.825046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:13.826363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:13.831356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:13.833721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 81/164] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:13.840159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.844778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 84/165] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:13.850246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:13.934863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.003060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24c811-3356-4321-94f4-5be3c06425a1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:14.084597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:14.086569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.087594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.090355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9c4-44dc-485b-a838-36c6fb0d1032 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:14.090900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.093562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:14.095284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5208e-5138-4078-ace0-ca3bb67f4ee0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:14.097193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 85/166] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:14.098857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527750-5797-46b4-9387-9dd0ece2b16a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.100231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527750-5797-46b4-9387-9dd0ece2b16a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.100704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527750-5797-46b4-9387-9dd0ece2b16a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.106813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 86/167] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:14.107803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527750-5797-46b4-9387-9dd0ece2b16a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:14.179103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:14.180135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.180614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.181091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.268633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.339200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:14.344216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71d074-37f9-4985-b6ae-d7c474dab7d4 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:14.364699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527750-5797-46b4-9387-9dd0ece2b16a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:14.456874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:14.457534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.457817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.458091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.477871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.562229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 82/168] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 00:21:14.563929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8f27-b023-42ef-ae79-f1dae16f6315 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.564411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8f27-b023-42ef-ae79-f1dae16f6315 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.564879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8f27-b023-42ef-ae79-f1dae16f6315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.582724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:14.584180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-809e8f27-b023-42ef-ae79-f1dae16f6315 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:14.584576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0bf13-5e0a-4a32-a5a7-fa1cd0f4345d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:14.585973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-809e8f27-b023-42ef-ae79-f1dae16f6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:14.654686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.661271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 83/169] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:14.662802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c569f-5b63-46ac-8239-0ecf6b5d8702 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.663423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c569f-5b63-46ac-8239-0ecf6b5d8702 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.663887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c569f-5b63-46ac-8239-0ecf6b5d8702 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.665427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 84/170] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:14.670506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4e03-6619-4d23-86fa-c2a8b044d89a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.671077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4e03-6619-4d23-86fa-c2a8b044d89a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.671553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4e03-6619-4d23-86fa-c2a8b044d89a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.673080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 85/171] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:14.673981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e4e03-6619-4d23-86fa-c2a8b044d89a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:14.716153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 87/172] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:14.717677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacef22c-fbd5-414c-b96c-5372ab1fa993 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.717973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacef22c-fbd5-414c-b96c-5372ab1fa993 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.718249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacef22c-fbd5-414c-b96c-5372ab1fa993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.740244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-eacef22c-fbd5-414c-b96c-5372ab1fa993 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:14.741730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eacef22c-fbd5-414c-b96c-5372ab1fa993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:14.787003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.806042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e4e03-6619-4d23-86fa-c2a8b044d89a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:14.836541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 88/173] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:14.840111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.840528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.840928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.860537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:14.862769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:14.864721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:14.865008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] PATH_INFO: `/v3/projects/7980010e74fe43cea131486b5e951608` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:14.866400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] RBAC: Authorizing `identity:get_project(project_id=7980010e74fe43cea131486b5e951608)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:14.868155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:14.870299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 86/174] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:14 2022] GET /identity/v3/projects/7980010e74fe43cea131486b5e951608 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:14.871187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:14.871593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:14.872521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:14.942016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 89/175] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:14.943128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:14.946733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:14.986725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39641616-56de-4fe8-a928-92eb1cd78309 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.027182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.028476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.028957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.029449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.043003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:15.047869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.051847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046f55f-6e66-475c-9eff-9b3709ab6736 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:15.054474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:15.055561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:15.056381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:15.059272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 87/176] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:15.061159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.088238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.089595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.089874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.090277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.091369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.092823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.095529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 90/177] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:14 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.095639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.175252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f233-154b-4d97-98d1-fc8f6dc320d3 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.202176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.208653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dececd-8a4c-48dc-9227-c33ad3b58e67 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.211337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.212127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.212403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.212670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.223504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:15.231652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:15.232550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:15.233372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 88/178] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:15.234026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.244516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.245703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.245981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.246252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.247107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=7980010e74fe43cea131486b5e951608, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:15.247416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.248844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.254542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 91/179] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=7980010e74fe43cea131486b5e951608&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.255322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.292432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.359131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2145361-48fe-444a-9b65-80a58ac05833 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.371074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da13108-c019-4848-a70d-df2484eca3b2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.396959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.397794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.398069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.398354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.409455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:15.412663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.413336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.413631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.413890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.415424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:15.416225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.417165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fb2e2-f637-4043-ae76-ba0bb48ecc95 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a7310c07-0397-4076-80a7-854a95b4be53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:15.417301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:15.418124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:15.419160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 89/180] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:15.423997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c2c0f-5f89-42f5-b049-de6aab695727 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.424534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c2c0f-5f89-42f5-b049-de6aab695727 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.424814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c2c0f-5f89-42f5-b049-de6aab695727 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.426141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 90/181] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:15.427155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c2c0f-5f89-42f5-b049-de6aab695727 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.427701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 92/182] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.428989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068309e-f2b8-4196-8f37-182f2f5cb7e3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.429274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068309e-f2b8-4196-8f37-182f2f5cb7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.429550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068309e-f2b8-4196-8f37-182f2f5cb7e3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.430812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 93/183] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:15.438598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068309e-f2b8-4196-8f37-182f2f5cb7e3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.497908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.539508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.550931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068309e-f2b8-4196-8f37-182f2f5cb7e3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.551058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c2c0f-5f89-42f5-b049-de6aab695727 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.583861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.584045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.584541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.584893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.584976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.585060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.585127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.585192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.585871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:15.585969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:15.586182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.586269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.587587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7f737-f641-48f1-b1d8-0b0add88a30f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.587681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.593592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 91/184] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.594298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.600557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 94/185] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.600557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.600557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.600557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.615950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:15.617088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:15.690323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 95/186] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:15.691042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.721220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9fd8e-426a-43c2-acdc-7a45ea2adf5c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.736866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.765408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.766086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.766369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.766640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.767430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:15.767738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.769127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8455f-3880-435f-9021-7473b23b14ba None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.778741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 92/187] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.780007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389c69-e153-4329-a282-edba322e0260 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.780291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389c69-e153-4329-a282-edba322e0260 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.780573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389c69-e153-4329-a282-edba322e0260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.800278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-16389c69-e153-4329-a282-edba322e0260 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:15.801732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-16389c69-e153-4329-a282-edba322e0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:15.805801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:15.823252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b25b2-fa7d-4360-a18a-0cedc43767b3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.858174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:15.859081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:15.859361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:15.859633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] PATH_INFO: `/v3/users/d7a6b76f2705487dafd371ca25ccca47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:15.860823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a6b76f2705487dafd371ca25ccca47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:15.862266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:15.863919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 96/188] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/users/d7a6b76f2705487dafd371ca25ccca47 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:15.864666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.879594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 93/189] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:15.880368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16389c69-e153-4329-a282-edba322e0260 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:15.970541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd87ea0-2441-4004-90b9-39f0a6676524 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.977952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16389c69-e153-4329-a282-edba322e0260 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:15.981103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.004204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.005070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.005353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.005622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] PATH_INFO: `/v3/users/8a66399190ef4fb898e50aa8a146b4a5` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.006833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=8a66399190ef4fb898e50aa8a146b4a5)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.008270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.009892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 97/190] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/users/8a66399190ef4fb898e50aa8a146b4a5 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:16.010665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.016341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.017135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.017412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.017685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.018889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.020329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.022054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 94/191] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:16.022708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.032089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.110067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73a1a8-0758-4c6f-b724-b41840071ef5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.119021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfae36-71bb-4f19-823e-dbfb03c36a47 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.143287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.144085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.144368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.144638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.153521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.154166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.154440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.154703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.161098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.162581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.164309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 98/192] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:16.165065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.201805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.229885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.231432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.242940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.266280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90030d4d-b335-4ab6-a4b2-281c7a7a5df5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.299284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.300076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.300353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.300620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.310432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:16.318453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:16.319429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:16.320220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 99/193] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:16.320880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.331859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 95/194] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:16.332466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.399831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.425351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174254-ae44-4b33-98e0-f1d66ebe8968 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.431024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926e17f-5034-4fe4-958e-cc07b72a5907 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.457571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.458214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.458492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.458764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.464820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.465467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.465745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.466014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.466719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.467409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.468186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.490258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 96/195] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:16 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:16.501691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.539046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.540537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.617346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fdf20-5282-4201-8cab-a4e0736ab0ac None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.648289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.653355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 100/196] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:16.654726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.661967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.662628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.663008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.663250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.703960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.750350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.751956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.767574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da22f-5e7e-4d8a-9a22-ca6009344f46 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:16.814195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:16.815029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:16.815593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:16.815593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:16.816316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:16.816622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:16.817997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:16.828895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 101/197] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:16.831359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.881126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 97/198] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:16.883678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:16.916613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.968434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:16.971616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9bf78-37e9-4fbd-af61-a6f21ea20d02 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.008963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.009599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4a3c-6d5d-4f94-95fa-2d80a69a5c76 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.010078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.010513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.010949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.012255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.013826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.015831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 102/199] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.021424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.054039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.055880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.056321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.056740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.059898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.061503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.063696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 98/200] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:16 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.065147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.158314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b247913-fc5b-4931-972c-5ecc63d7e1a7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.177527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:17.187753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb8850-0f56-449d-9051-3293e5508671 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.203611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.204651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.205027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.205397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.206900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.208472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.210604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 103/201] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.211460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.237410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.238419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.238868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.239152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.240611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.242799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.246627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 99/202] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.247583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.253667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:17.342731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ca51-1f5b-4fc8-ad2f-5c9022ea0aa7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.361487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2c64-9fee-4473-9508-379043ef9c85 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.391349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.392007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.392288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.392561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.393397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=d7a6b76f2705487dafd371ca25ccca47) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:17.393712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.395224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.402221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 104/203] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/role_assignments?user.id=d7a6b76f2705487dafd371ca25ccca47&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=87cd2d147f344fa99848788e1a735d8f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.405140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.406085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.406572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.407068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.408214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, scope.project.id=23fdf0097d574eceb9554720e660ec1c, user.id=8a66399190ef4fb898e50aa8a146b4a5) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:17.408743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.409443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.410798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.420146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 100/204] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/role_assignments?user.id=8a66399190ef4fb898e50aa8a146b4a5&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=205663cf9d3242d4a020abca619acbee => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.426809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:17.488262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:17.621020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6053cf-64b2-43c6-8855-f040245b1957 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.700232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:17.704883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7dd1a-2fb5-4858-a093-465074913f8e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:17.707737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.708412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.708690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.708963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.709888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=49a2f875818e4387b83a8127d90139df) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:17.710201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.712681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.725728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 101/205] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/role_assignments?user.id=49a2f875818e4387b83a8127d90139df&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:17.757299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:17.757905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:17.758188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:17.758483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:17.854267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:17.855796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:17.978942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:17.983339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 105/206] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:17.989244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:18.088046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:18.098308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf15c-d1fd-49a3-a73f-b52c7912e813 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:18.134662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:18.136474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.136957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.137450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.138905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:18.140551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d140c07-8598-47e4-96f6-9928d9ba620f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:18.142830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 102/207] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:17 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:18.150406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:18.262543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dd071-3e47-4260-b6b2-2cc06b9d586f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:18.297814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:18.298595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.298768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.299055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.299907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=49a2f875818e4387b83a8127d90139df) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:18.300222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:18.301674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187cb1df-91a7-4ffd-abce-6e61246607aa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:18.308543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 106/208] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:18 2022] GET /identity/v3/role_assignments?user.id=49a2f875818e4387b83a8127d90139df&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=87cd2d147f344fa99848788e1a735d8f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:18.927883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c6cbf-0012-4d5c-8880-968cb0ce3a5b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.927883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c6cbf-0012-4d5c-8880-968cb0ce3a5b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.927883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c6cbf-0012-4d5c-8880-968cb0ce3a5b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.927883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 103/209] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:18.933682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.934477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.935003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.941102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfbcd-0bb3-45c3-a341-bcf9b55bc6f7 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.941710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfbcd-0bb3-45c3-a341-bcf9b55bc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.942177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cfbcd-0bb3-45c3-a341-bcf9b55bc6f7 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.944143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 104/210] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:18.950362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:18.951044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:18.951530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:18.958656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:18.959855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:18.972804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:18.974268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:19.046521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:19.048546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 105/211] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:19.062101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 107/212] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:19.073716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.084654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.193555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:19.203943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfc746-6c94-4324-b703-35699d612d32 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.214902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bf386-81de-491c-8eee-8cff3cb42bc0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.241482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30098e61-b043-4245-989b-ce458430d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.242506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098e61-b043-4245-989b-ce458430d006 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.243081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098e61-b043-4245-989b-ce458430d006 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.243559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098e61-b043-4245-989b-ce458430d006 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.244954 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30098e61-b043-4245-989b-ce458430d006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:19.246613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30098e61-b043-4245-989b-ce458430d006 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:19.248479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 106/213] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:19.255972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30098e61-b043-4245-989b-ce458430d006 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.257833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.258625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.258930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.259207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.273078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:19.284224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:19.285112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:19.285899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:19.290284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 108/214] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:19.291209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.364625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:19.374261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30098e61-b043-4245-989b-ce458430d006 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.415462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf0ed-5cf9-46dc-b0ce-48cfd0c6c0b3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.427125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.427802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.428087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.428366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.429334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:19.431170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:19.461083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 107/215] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:19 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:21:19.462193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:19.470814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:19.483640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 109/216] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:19.492855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.514669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:19.610196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd7c80-1a8c-4b7d-bfcd-4845f404de1b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.648516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.649741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.650257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.650905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] PATH_INFO: `/v3/users/a854e6ad1df24fb894d56f4106e86cde` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.665896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a854e6ad1df24fb894d56f4106e86cde)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:19.667739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:19.669737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 108/217] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:19 2022] GET /identity/v3/users/a854e6ad1df24fb894d56f4106e86cde => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:19.675233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:19.784020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff002f-74b9-4c73-89c8-839c35ef0fcc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:19.811645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:19.823962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:19.824832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:19.825258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:19.825699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:19.905520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:19.907258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab44ad-f664-47a1-9a92-5fb939fab9e1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:20.018962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 110/218] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:20.024249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.046493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9122c-bbc6-44c2-8a20-6ef94b09c412 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.047021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9122c-bbc6-44c2-8a20-6ef94b09c412 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.047437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9122c-bbc6-44c2-8a20-6ef94b09c412 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.049020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 111/219] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:20.059143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.059635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.060074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.088523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:20.089945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:20.106846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.107996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.142397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ad4b3-be7c-4808-99ab-c5edb07c62b1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.186805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.186805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.186805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.186805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.187470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:20.187470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:20.187719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 109/220] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:20.191454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 112/221] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:20.199848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.214767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.377633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d37669-64b0-4f23-bc10-9f07b3fa37df None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.390914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab876-c15e-4e62-baab-1785c1b084e9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.422047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.423068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.423555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.424003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.425048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=a854e6ad1df24fb894d56f4106e86cde) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:20.428950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:20.430752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b098be-eb12-4e42-8ffb-63ba4780847e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:20.441458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 110/222] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/role_assignments?user.id=a854e6ad1df24fb894d56f4106e86cde&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:20.468018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.469165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.469620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.470063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.496519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b09fb-8b6d-4e6c-8a08-6235e0f702dd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.496807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b09fb-8b6d-4e6c-8a08-6235e0f702dd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.497082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b09fb-8b6d-4e6c-8a08-6235e0f702dd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.498604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 111/223] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:20.501095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:20.508576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:20.510280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:20.511214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:20.512004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:20.514985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 113/224] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:20.519507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.519507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.520822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.535163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.536500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.567984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:20.569204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:20.679529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 112/225] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:20.680054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dc6d7-579b-4af8-87dd-8112e5c36e6b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.708464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.730483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.731184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.731461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.731729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.732480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:20.732784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:20.734173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:20.751557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 114/226] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:20.758822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.827179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.860206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72819d5f-c034-41ae-ab7f-d6d535c90099 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.907539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.908408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.908682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.908953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.913046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:20.914499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:20.917653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 115/227] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:20.927773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:20.933161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d7ee-830f-4bfb-92e9-7b702db4336d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:20.970379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:20.971549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:20.972104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:20.972590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:20.974724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:20.985390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:20.994701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:20.995733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:20.996564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:20.999710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 113/228] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:21.007101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:21.121318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf174-08e3-4fee-9741-e124c025a8ea None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.130164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510252a3-eb82-40cd-b4d3-e4ed61f8bb32 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.160076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.161113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.161591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.162092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.163117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:21.163650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.165341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.174961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 114/229] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:21.179916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:21.199929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.201170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.201646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.202109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.205970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.208490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.218993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 116/230] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:20 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:21.220441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:21.225116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:21.325177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:21.356928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c7ee8-ae31-40fd-8b7a-8065615e7319 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.396107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.397857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.398339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.398833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/49a2f875818e4387b83a8127d90139df/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.401905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, user_id=49a2f875818e4387b83a8127d90139df, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.404560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.412699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce4c1-8d0d-417b-a666-04e07a2a4930 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.427429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 117/231] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:21 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/49a2f875818e4387b83a8127d90139df/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:21:21.462144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.463143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.463431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.463703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.464913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.466430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.468427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 115/232] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:21 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:21.473725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:21.545101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:21.587197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc423e-38ef-4f41-85ce-cc5b6fd81fc5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.624555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.625432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.625708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.625984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.627317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.628852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896081f-f8ef-4a8d-8628-f3ddcf8bef82 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.630610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 118/233] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:21 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:21.636366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:21.750932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f51238-870d-41e7-ae0b-16eab22c6fdd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:21.775513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:21.786754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:21.787751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:21.788030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:21.788302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/49a2f875818e4387b83a8127d90139df/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:21.790699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, user_id=49a2f875818e4387b83a8127d90139df, role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:21.793000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77579-0f55-429b-b058-941bcb778db5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:21.812213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 116/234] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:21 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/49a2f875818e4387b83a8127d90139df/roles/87cd2d147f344fa99848788e1a735d8f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:22.292253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ad85-529c-45ee-aefd-5500d6933cc3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.292959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ad85-529c-45ee-aefd-5500d6933cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.292959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9ad85-529c-45ee-aefd-5500d6933cc3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.294167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 119/235] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:22.299497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.299933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.300249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.309792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede9e81-911c-4e23-af8c-3f0bc44a5ea3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.310086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede9e81-911c-4e23-af8c-3f0bc44a5ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.310368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede9e81-911c-4e23-af8c-3f0bc44a5ea3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.311996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 120/236] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:22.316975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.317260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.317533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.318817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:22.320101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:22.345038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:22.346242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:22.453968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 121/237] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:22.460373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 117/238] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:22.486133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:22.496704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:22.591755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:22.633846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b296-3fac-4e09-a768-6d85a8fbf0b3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:22.671280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:22.672601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.673077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.673559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.675086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:22.676738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:22.682855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 122/239] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:22.684830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:22.776784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969e9d-ff79-4081-bf41-26867c14c6e7 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:22.786687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9334-e7e2-493e-b85b-96a60d891ffb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:22.815858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:22.820353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:22.825508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:22.826436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.826966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.827427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.828318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:22.829933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:22.855839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 123/240] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:22 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:21:22.883801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:22.884703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.885600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.885889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.907741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-dc06ea2f-a4e2-44ff-9088-faffbf7e4fb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:22.915809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:22.916812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:22.917601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 118/241] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:22.924064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:22.934947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8ab3-be22-4eb3-b923-7078af87cefd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.934947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8ab3-be22-4eb3-b923-7078af87cefd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.934947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8ab3-be22-4eb3-b923-7078af87cefd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.936073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 119/242] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:22.941206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:22.941495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:22.941776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:22.959957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:22.961152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:23.086447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 120/243] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:23.110561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.115907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c458c-93d6-487f-bf31-d5948527cbd5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.164241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.164918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.165330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.165585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.166946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:23.167259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.168825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.177997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 124/244] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:23.185088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.238201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd37b5-3f4b-46d3-880d-1c0a0c12219f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.240728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.242676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.285640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.286451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.286733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.287038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.298158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:23.306668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:23.308577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:23.309369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:23.309969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31ab81-1899-48b6-9c58-47df66c1915d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.310296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 121/245] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:23.323920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.347367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.348263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.348597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.348924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] PATH_INFO: `/v3/users/48af6d45930c4854916862dcbe0fdee0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.365323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] RBAC: Authorizing `identity:get_user(user_id=48af6d45930c4854916862dcbe0fdee0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.366874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.368774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 125/246] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/users/48af6d45930c4854916862dcbe0fdee0 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:23.375809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.460534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f51bc-78c3-4f05-a65a-afb07c9cda06 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.500875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3426683-6a54-4c9c-afa4-0b75f7cdeda9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.508672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.509553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.510017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.510130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.510523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.511476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:23.511947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.513501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.522995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 122/247] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:23.529724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.546583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.547484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.547748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.548025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.566287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.642991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.645432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.665579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c89bb-b75e-4407-b25e-e3cf87ce7f19 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.702409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.703267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.703635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.703815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] PATH_INFO: `/v3/users/a854e6ad1df24fb894d56f4106e86cde` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.705077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] RBAC: Authorizing `identity:get_user(user_id=a854e6ad1df24fb894d56f4106e86cde)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.706573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.708647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 123/248] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/users/a854e6ad1df24fb894d56f4106e86cde => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:23.719760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.769417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 126/249] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:23.775572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.785261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.888590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d8c-3d55-4ead-8735-8d83694114ab None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:23.926402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:23.927741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:23.928227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:23.928714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:23.930140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:23.931860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:23.934286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 127/250] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:23.939299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:23.941124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:23.946021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ac63-15eb-4553-91bf-4d8a8b0cd47a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:24.007515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:24.008757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.008757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.009059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.017133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:24.018616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:24.020986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 124/251] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:24.026865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:24.027344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:24.056446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec5a1-aa96-4d7a-8dff-ad162b8a9e5d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:24.093503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:24.094650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.095251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.095721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.096768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=48af6d45930c4854916862dcbe0fdee0) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:24.097290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:24.099008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34733477-51c3-46c5-9ae3-ed694a90229d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:24.105773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 128/252] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:23 2022] GET /identity/v3/role_assignments?user.id=48af6d45930c4854916862dcbe0fdee0&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=205663cf9d3242d4a020abca619acbee => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:24.204551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0c8d-136e-426f-ab78-c040f5a42d40 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:24.260880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:24.261849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.262126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.262393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/a854e6ad1df24fb894d56f4106e86cde/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.264440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, user_id=a854e6ad1df24fb894d56f4106e86cde, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:24.265897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e304472-486b-4c7a-ace3-8db03fcb3147 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:24.296225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 125/253] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:24 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/a854e6ad1df24fb894d56f4106e86cde/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:24.481453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5d3e0-2faf-495b-9768-b9f091034ad9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.482393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5d3e0-2faf-495b-9768-b9f091034ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.482915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5d3e0-2faf-495b-9768-b9f091034ad9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.484524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 129/254] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:24.491710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a38d8-7fa4-481d-a441-7634e3c16f61 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.491710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a38d8-7fa4-481d-a441-7634e3c16f61 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.491710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a38d8-7fa4-481d-a441-7634e3c16f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.518247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3c2a38d8-7fa4-481d-a441-7634e3c16f61 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:24.519479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3c2a38d8-7fa4-481d-a441-7634e3c16f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:24.552741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:24.621549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 126/255] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:24.638931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418398c4-33f3-4084-b62a-1fb462bb0fb9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.639554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418398c4-33f3-4084-b62a-1fb462bb0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.640026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418398c4-33f3-4084-b62a-1fb462bb0fb9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.641855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 130/256] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:24.646977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.647263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.647535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.665345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418398c4-33f3-4084-b62a-1fb462bb0fb9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:24.670819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:24.671997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:24.754707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 127/257] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:24.792575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:24.888375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418398c4-33f3-4084-b62a-1fb462bb0fb9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:24.969430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:24.970247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:24.970528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:24.970817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:24.984406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:24.985498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:24.994790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:25.004940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:25.005852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:25.006662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 131/258] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:25.014371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:25.024958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f45f7b-fb49-4fc3-b11d-c3be3288b00b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.159763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.160595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.160876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.161148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.167764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2044fef-bf59-4f31-8d52-b5760c4be22a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.199687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:25.215934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:25.216842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:25.217650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 128/259] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:25.222755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:25.241011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.241965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.242414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.242694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.243499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:25.243808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:25.246168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:25.266551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 132/260] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:25.271650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:25.369840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:25.446228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a207e-de5e-4577-b214-704e8a628d17 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.472111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a17cb-68bb-41e0-9cb2-3e03c68885a2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.513140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.514228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.514516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.514811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.516201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:25.517703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:25.519331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 133/261] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:25.524992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:25.543192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.543865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.544148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.544426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.545189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:25.545494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:25.546981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:25.562612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 129/262] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:25.571795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:25.584080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:25.773864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaae98-6f97-4949-b982-73a9e8a4e96e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.834478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d044c2-bc73-4430-a052-c30a6837714f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:25.871863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:25.903610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.904278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.904560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.904833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.928011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:25.928904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:25.929187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:25.929461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] PATH_INFO: `/v3/users/49a2f875818e4387b83a8127d90139df` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:25.930801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] RBAC: Authorizing `identity:get_user(user_id=49a2f875818e4387b83a8127d90139df)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:25.932512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:25.934180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 130/263] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/users/49a2f875818e4387b83a8127d90139df => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:25.940849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:26.075863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.077275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.187214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:26.244006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a3474-eb8c-4ca3-b21e-4e72dfe13591 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:26.287091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 134/264] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:26.295813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:26.332500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:26.333098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.333388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.338797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.341278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:26.417804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22a08d-6ff3-43af-ad9e-4093b3af7f32 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:26.448909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.450669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.462290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:26.463233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.463509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.463781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.465001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.466571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.468426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 135/265] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:26 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:26.481482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:26.568182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:26.611075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 131/266] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:26.620471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:26.735919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963653-6002-4784-8921-2c94f3b88c92 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:26.774448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:26.775437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.776191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.776465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.777856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.779426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25039c89-c819-47cd-98a1-753fa9e5edfb None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.781162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 132/267] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:26 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:26.782553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eccb9-a610-4607-be95-e86a15274bca None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.782851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eccb9-a610-4607-be95-e86a15274bca None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.783311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6eccb9-a610-4607-be95-e86a15274bca None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.784569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 133/268] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:26.789742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6eccb9-a610-4607-be95-e86a15274bca None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:26.826330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:26.837280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:26.860812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e44df-a3e7-4766-b59f-7ed1bfdef1d4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:26.902609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6eccb9-a610-4607-be95-e86a15274bca None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:26.936345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:26.937007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.937288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.937564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.938403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=49a2f875818e4387b83a8127d90139df) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:26.938713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.940300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0776949-f248-483d-a355-2b2c93153fc0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.943041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:26.944257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.944587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.945148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.945997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=49a2f875818e4387b83a8127d90139df) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:26.946310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:26.947941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 136/269] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:26 2022] GET /identity/v3/role_assignments?user.id=49a2f875818e4387b83a8127d90139df&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:26.948378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f878625-5149-4ba5-98b4-46499d0ffc95 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:26.955005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 134/270] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:26 2022] GET /identity/v3/role_assignments?user.id=49a2f875818e4387b83a8127d90139df&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=87cd2d147f344fa99848788e1a735d8f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:26.956326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:26.956689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:26.957039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:26.983219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:26.984448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:27.091074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad67b99-ed41-4ede-885a-6020ea28fcce None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.091709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad67b99-ed41-4ede-885a-6020ea28fcce None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.092177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad67b99-ed41-4ede-885a-6020ea28fcce None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.093796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 135/271] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:27.099991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d440fc-b3f5-49a9-85f2-bb20ba91fd4f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.100323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d440fc-b3f5-49a9-85f2-bb20ba91fd4f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.100541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d440fc-b3f5-49a9-85f2-bb20ba91fd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.101413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 137/272] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:27.136562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-65d440fc-b3f5-49a9-85f2-bb20ba91fd4f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:27.137810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-65d440fc-b3f5-49a9-85f2-bb20ba91fd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:27.146451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.261316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 136/273] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:27.263393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2f4-7adc-4803-be10-0da3883d4b6c None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.263700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2f4-7adc-4803-be10-0da3883d4b6c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.263985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2f4-7adc-4803-be10-0da3883d4b6c None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.266306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 137/274] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:27.272358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.272788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.273074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.301178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:27.302704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:27.315934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.335576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b09df-b3d3-498f-b6f4-5008b72dc196 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.348651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.385777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.386584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.386890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.387160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.388346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:27.389844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:27.391663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 138/275] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:27.392550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.398093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 138/276] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:27.398814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.509726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25a322-63ec-4322-ba4a-cf21132e5c95 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.514822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e7c4-e03e-4c41-a93c-93918a8c7624 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.542822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.543612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.543890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.544162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.547907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.548569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.548851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.549151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.549852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:27.551355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:27.554127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f4e4f59b-e796-44ca-9455-411934688775 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:27.563115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:27.564132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:27.565009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 139/277] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:27.571076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9540c56-ca31-4ddd-9e37-f4720486c71d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.571076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9540c56-ca31-4ddd-9e37-f4720486c71d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.571076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9540c56-ca31-4ddd-9e37-f4720486c71d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.571076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 140/278] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:27.571076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9540c56-ca31-4ddd-9e37-f4720486c71d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.585798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 139/279] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:27 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:27.585798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.601956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.602203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.694300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061876e9-6056-4ac7-82f6-88c7bb6e8efc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.698509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9540c56-ca31-4ddd-9e37-f4720486c71d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.733825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.734492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.735311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.735311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.735795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:27.736880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:27.736992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.737475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.737571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251853d0-7c29-43a6-a62b-67717ff76865 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:27.737777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.738055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.749078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 140/280] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:27.749966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.750257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.750519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.750733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:27.759411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:27.761800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 141/281] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:27.762814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.770244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:27.771508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:27.857454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.858416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 141/282] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:27.859461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.879690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:27.884964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e62a-9e34-4e9f-8ea0-0782682f341b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:27.929668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:27.930537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:27.930935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:27.931215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] PATH_INFO: `/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:27.946404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] RBAC: Authorizing `identity:get_user(user_id=bb9b3a0e7cca441ea81718d4ba763d4e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:27.947941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:27.949831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 142/283] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:27.950767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:27.983159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb249ed-cc2f-4c9c-b202-beca6cb26eaa None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.029338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.029925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.030209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.030724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.031514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:28.032693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.034076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.049301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 142/284] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:28.049301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.073419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352f6a9-c976-4538-867e-6fbb8af46d4f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.117869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.119499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.119834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.120106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.124053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.137476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:28.142939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:28.146433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:28.147378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:28.148165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:28.151285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 143/285] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:28.152586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.175280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811f71-767f-4aaf-b589-d5513332d7e2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.213430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.214120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.214623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.215071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.216222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, user.id=bb9b3a0e7cca441ea81718d4ba763d4e, scope.system=all) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:28.216560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.219378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.227350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 143/286] 10.222.0.33 () {56 vars in 1308 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/role_assignments?user.id=bb9b3a0e7cca441ea81718d4ba763d4e&scope.system=all&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:28.228015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.348915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645a10-bf7d-485a-a23b-12e71bf5c0f8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.359163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d4fa4-a1a6-45f5-b6a4-986563e56821 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.409064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.409963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.410250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.410527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.412637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:28.412952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.414339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.420238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.426026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 144/287] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:28.426260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.427468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.427943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.428425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] PATH_INFO: `/v3/users/48af6d45930c4854916862dcbe0fdee0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.430935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.431780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] RBAC: Authorizing `identity:get_user(user_id=48af6d45930c4854916862dcbe0fdee0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.433421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.436297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 144/288] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/users/48af6d45930c4854916862dcbe0fdee0 => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:28.441686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.516851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.560867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7726571-15f4-43b2-85ac-28a31416b025 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.618764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0f55-302f-4f51-9754-9ad267e33402 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.625389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.626318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.626598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.626891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.628129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.629578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.631310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 145/289] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:28.637840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.743675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef576136-f5be-4b28-a676-442ea9260265 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.755480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.756371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.756650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.756918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] PATH_INFO: `/v3/users/a854e6ad1df24fb894d56f4106e86cde` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.758163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a854e6ad1df24fb894d56f4106e86cde)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.759708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.761855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 145/290] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/users/a854e6ad1df24fb894d56f4106e86cde => generated 297 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:28.771697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:28.775626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.783803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.784819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.785094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.785784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/48af6d45930c4854916862dcbe0fdee0/roles/205663cf9d3242d4a020abca619acbee` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:28.788696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, user_id=48af6d45930c4854916862dcbe0fdee0, role_id=205663cf9d3242d4a020abca619acbee)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:28.790423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:28.805339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 146/291] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:21:28 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/users/48af6d45930c4854916862dcbe0fdee0/roles/205663cf9d3242d4a020abca619acbee => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:28.910011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:28.951851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb109ae4-ae12-45b4-9e5d-2700a7a3b0a8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:28.992541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f687d-7946-497f-93a4-b554af28919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:28.993191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f687d-7946-497f-93a4-b554af28919c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:28.993474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f687d-7946-497f-93a4-b554af28919c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:28.993753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f687d-7946-497f-93a4-b554af28919c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:29.066545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f687d-7946-497f-93a4-b554af28919c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:29.068066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f687d-7946-497f-93a4-b554af28919c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:29.136008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:29.172164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 146/292] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:29.177830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:29.325485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff530f-be5e-4975-a015-bf160d390160 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:29.377995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:29.378932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:29.379217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:29.379492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:29.380687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:29.382121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7158a-71fe-4f21-a70a-7d8dceedf33c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:29.384158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 147/293] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:29 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:29.390061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f687d-7946-497f-93a4-b554af28919c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:29.552719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f687d-7946-497f-93a4-b554af28919c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:29.594211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:29.595270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:29.595748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:29.596232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:29.597287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=a854e6ad1df24fb894d56f4106e86cde) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:29.597794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:29.599443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79c43d-6d06-48e8-b72f-60c1f165b79f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:29.611125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 147/294] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:29 2022] GET /identity/v3/role_assignments?user.id=a854e6ad1df24fb894d56f4106e86cde&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:30.162404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6baeed-88f9-4829-97c6-8145ee194891 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.164354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6baeed-88f9-4829-97c6-8145ee194891 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.164354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6baeed-88f9-4829-97c6-8145ee194891 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.164872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 148/295] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:30.166995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b011-9e69-4826-809e-7fad7bfe3cec None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.167819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b011-9e69-4826-809e-7fad7bfe3cec None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.168283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b011-9e69-4826-809e-7fad7bfe3cec None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.170899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.171277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 148/296] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:30.171759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.172285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.176234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.176518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.176793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.194124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:30.195972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:30.199801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:30.201234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:30.248643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.279770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 149/297] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:30.286951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.301942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 149/298] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:30.309554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.333014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.438184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a6f6-6896-4588-9baf-63b5212de757 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:30.467484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbcba4-26c8-4bfb-a126-24b8e3802e1f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:30.481560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:30.482448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.482822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.483176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.484422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:30.485987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:30.491194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 150/299] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:30.493949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.518920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:30.520620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.521322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.521781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.539492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:30.551076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:30.552193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:30.553156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:30.556120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 150/300] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:30.560319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.562734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.596335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.606945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3af247-c599-443e-87b1-3e98c6e27295 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:30.642707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:30.643617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.643999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.644546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] PATH_INFO: `/v3/groups` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.645209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:30.646763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700fd5-ee98-4e9b-b78e-c6a58edc86ec None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:30.668896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 151/301] 10.222.0.33 () {60 vars in 1134 bytes} [Wed Jun 15 00:21:30 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:30.700703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64794402-a5d3-414b-b04d-50c3fa7955fa None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:30.741243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f11eb-bd91-4189-8387-af63b2eff655 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.741554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f11eb-bd91-4189-8387-af63b2eff655 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.741801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f11eb-bd91-4189-8387-af63b2eff655 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.743442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 152/302] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:30.751262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.751637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.751921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.752668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:30.753570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.754083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.754554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.755615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:30.756133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:30.757778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:30.767334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 151/303] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:30.773658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.775016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:30.776275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:30.859622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 153/304] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:30.881735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.887939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.907171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76fdb-e770-4b1d-aeb9-4e85f17eee91 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:30.944263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:30.948005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:30.948898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:30.949181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:30.949449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] PATH_INFO: `/v3/users/6ca9717358f74a95b28ee2aec7ff64db` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:30.964355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] RBAC: Authorizing `identity:get_user(user_id=6ca9717358f74a95b28ee2aec7ff64db)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:30.965789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:30.967859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 152/305] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/users/6ca9717358f74a95b28ee2aec7ff64db => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:30.973773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:30.996555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187dce47-8d32-46dc-b349-5cd83154bc36 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.033738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.034599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.034949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.035226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.046600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:31.054729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:31.055726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:31.056569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:31.061159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 154/306] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:31.062030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:31.105184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114fee-9eec-42d4-8dad-73d47e5ca80a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.150971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.152352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.152768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.153538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.154802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, user.id=6ca9717358f74a95b28ee2aec7ff64db, scope.system=all) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:31.155227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:31.157531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:31.164989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 153/307] 10.222.0.33 () {56 vars in 1308 bytes} [Wed Jun 15 00:21:30 2022] GET /identity/v3/role_assignments?user.id=6ca9717358f74a95b28ee2aec7ff64db&scope.system=all&role.id=205663cf9d3242d4a020abca619acbee => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:31.184376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03467d90-700b-4194-bf9e-35dad3a723c7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.195602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:31.225568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.226282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.226573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.226888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.227735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:31.228054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:31.230363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4edb7f-684b-4d54-ac64-d376e09002fc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:31.241025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 155/308] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:31.248162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:31.267155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:31.367221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a12f8-f8eb-4a2a-a78b-5adbf080631e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.376323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547be5-fb43-4da7-98ee-b0dcaee77de3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.376607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547be5-fb43-4da7-98ee-b0dcaee77de3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.376882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61547be5-fb43-4da7-98ee-b0dcaee77de3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.378387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 156/309] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:31.383351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.383630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.383902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.403981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:31.405154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:31.406330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.407453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.407931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.408416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] PATH_INFO: `/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.409833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] RBAC: Authorizing `identity:get_user(user_id=bb9b3a0e7cca441ea81718d4ba763d4e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:31.411594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:31.413738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 154/310] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:31 2022] GET /identity/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:31.419345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:31.503178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 157/311] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:31.516862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:31.530609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:31.549673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd10555-24ac-40c9-9d47-57c592443db0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.596668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.597533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.597925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.598224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] PATH_INFO: `/v3/system/users/bb9b3a0e7cca441ea81718d4ba763d4e/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.600089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bb9b3a0e7cca441ea81718d4ba763d4e, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:31.602147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:31.622806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 155/312] 10.222.0.33 () {58 vars in 1334 bytes} [Wed Jun 15 00:21:31 2022] PUT /identity/v3/system/users/bb9b3a0e7cca441ea81718d4ba763d4e/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:21:31.651421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:31.655010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017118e6-caf8-4724-ae29-04bb995666ac None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.695009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.695815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.696090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.696362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.708805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:31.716769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:31.717749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:31.718562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:31.722805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 158/313] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:31.724239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:31.831930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d118f-31ec-4ecd-b04e-c6edc6971169 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:31.874324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:31.875055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:31.875340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:31.875615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:31.876389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:31.876698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:31.878129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:31.884441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:31.893086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 156/314] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:31.896063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:32.001862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff96-19ba-4f04-b0cf-01b8aeca4585 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:32.036477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:32.037409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.037690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.037962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] PATH_INFO: `/v3/users/48af6d45930c4854916862dcbe0fdee0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.039245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] RBAC: Authorizing `identity:get_user(user_id=48af6d45930c4854916862dcbe0fdee0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:32.041245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:32.046885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 159/315] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:31 2022] GET /identity/v3/users/48af6d45930c4854916862dcbe0fdee0 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:32.048785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:32.147938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12404ae-385a-4499-8c1e-3181b5738a89 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:32.198813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:32.198813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.198813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.200192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.270587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:32.275464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:32.276977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:32.380975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 157/316] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:32.386725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:32.417775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:32.492645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db913815-b9ab-4f77-be97-b6c6d95fbc88 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:32.533834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:32.534710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.535022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.535298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.536526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:32.537982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e446-16a6-4c9f-b44c-668ef9348957 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:32.542944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 160/317] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:32 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:32.546602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:32.644552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:32.647325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac23b1-2cc1-441e-a3b3-f3244c97b39d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:32.682615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:32.683261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.683542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.683816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.684651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, scope.project.id=9283a59b6fdf429b88bad064de7440f8, user.id=48af6d45930c4854916862dcbe0fdee0) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:32.684960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:32.686395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179483ff-bff0-49f7-9d7a-254a0e9cda8a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:32.692906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 158/318] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:21:32 2022] GET /identity/v3/role_assignments?user.id=48af6d45930c4854916862dcbe0fdee0&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=205663cf9d3242d4a020abca619acbee => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:32.975249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5015-0826-4084-90bf-1fe12877840c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.975590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5015-0826-4084-90bf-1fe12877840c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.975802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c5015-0826-4084-90bf-1fe12877840c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.977316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 161/319] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:32.982592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:32.982981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:32.983340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:32.999992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:33.001164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:33.069063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 159/320] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:33.091967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c5015-0826-4084-90bf-1fe12877840c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:33.188673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:33.196931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c5015-0826-4084-90bf-1fe12877840c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:33.232529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:33.233245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.233532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.233786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.234967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:33.236394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64279fbd-0886-4ef4-8455-846826c7d3c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:33.238208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 162/321] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:33.243450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:33.339807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:33.349381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3c1df-f00f-4d13-b3aa-af30d788f80a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:33.383722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:33.384958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.385218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.385484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] PATH_INFO: `/v3/groups` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.386202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:33.387558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320971-7f42-46be-bc0b-9d188351031f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.387985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:33.388074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320971-7f42-46be-bc0b-9d188351031f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.388151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320971-7f42-46be-bc0b-9d188351031f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.389626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 163/322] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:33.394568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.394937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.395185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.403708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 160/323] 10.222.0.33 () {60 vars in 1134 bytes} [Wed Jun 15 00:21:33 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:33.412961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:33.414188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:33.492336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 164/324] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:33.515195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:33.629429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451e7e-f7e8-45f7-acb5-a1f04372d544 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:33.667861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:33.668706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.668986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.669267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.679929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:33.688234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:33.689112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:33.689902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 161/325] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:33.697105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:33.795148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:33.804588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2fd4-db85-4c64-b3bf-1951156b1aec None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:33.855511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:33.856196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.856482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.856759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.857529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:33.857836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:33.859333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc920-b919-4b3d-bb4d-8b65647cc2d1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:33.869239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 165/326] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:33.874802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:33.891928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:33.908974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4178-5414-4396-a136-c3d074a1a708 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.909264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4178-5414-4396-a136-c3d074a1a708 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.909536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4178-5414-4396-a136-c3d074a1a708 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.911186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 166/327] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:33.926726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:33.927027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:33.927366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:33.952404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:33.953610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:34.001320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda3c4a-4737-460b-b236-cbc63625a79a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.036226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 167/328] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:34.040303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.042012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.042451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.042942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] PATH_INFO: `/v3/users/6ca9717358f74a95b28ee2aec7ff64db` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.044377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] RBAC: Authorizing `identity:get_user(user_id=6ca9717358f74a95b28ee2aec7ff64db)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:34.045873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:34.047811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 162/329] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:33 2022] GET /identity/v3/users/6ca9717358f74a95b28ee2aec7ff64db => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:34.053565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:34.063573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:34.148662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:34.155408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:34.167804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633b7ef-7b1c-4654-8160-879073ee751e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.176738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33fb0d-5863-4b6e-82dd-aae2c726954e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.208798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.210297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.210834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.211744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] PATH_INFO: `/v3/system/users/6ca9717358f74a95b28ee2aec7ff64db/roles/205663cf9d3242d4a020abca619acbee` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.214019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ca9717358f74a95b28ee2aec7ff64db, role_id=205663cf9d3242d4a020abca619acbee)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:34.216959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:34.222614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.227034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.227034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.227034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.238811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 168/330] 10.222.0.33 () {58 vars in 1334 bytes} [Wed Jun 15 00:21:34 2022] PUT /identity/v3/system/users/6ca9717358f74a95b28ee2aec7ff64db/roles/205663cf9d3242d4a020abca619acbee => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:34.245022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:34.253362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:34.254380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:34.255210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 163/331] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:34.261303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:34.380573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb503ce-2b21-4db1-b350-f9ebe2c5c5b4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.424985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.425747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.425930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.426206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.426996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:34.427301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:34.428678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:34.438354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 169/332] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:34.443852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:34.585804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f339a1e-309c-4bed-9c25-925eba269ff1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.624472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.625611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.626029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.626439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] PATH_INFO: `/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.627803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] RBAC: Authorizing `identity:get_user(user_id=bb9b3a0e7cca441ea81718d4ba763d4e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:34.629373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb78c5-043f-442b-8435-7daf46aa22f4 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:34.631542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 164/333] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:34 2022] GET /identity/v3/users/bb9b3a0e7cca441ea81718d4ba763d4e => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:34.640083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:34.656612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:34.749723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa32282-c716-4d9f-ad3a-83380348377f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:34.784636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:34.785612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:34.786032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:34.786435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:34.787435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] RBAC: Adding query filter params (role.id=b9e44997cbef45b98a11c68cadd4cbf3, user.id=bb9b3a0e7cca441ea81718d4ba763d4e, scope.system=all) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:34.787884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:34.789553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62432e94-2573-4089-874a-22e2162ea8b1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:34.802616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 170/334] 10.222.0.33 () {56 vars in 1308 bytes} [Wed Jun 15 00:21:34 2022] GET /identity/v3/role_assignments?user.id=bb9b3a0e7cca441ea81718d4ba763d4e&scope.system=all&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:36.162672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dc5c9-1989-48b9-81ea-254d34409f64 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.163219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dc5c9-1989-48b9-81ea-254d34409f64 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.163295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dc5c9-1989-48b9-81ea-254d34409f64 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.164757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 165/335] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:36.172062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.172514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.172588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.191721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:36.192913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:36.268954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:36.280486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 171/336] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:36.324542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dc5c9-1989-48b9-81ea-254d34409f64 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:36.338056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:36.444984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dc5c9-1989-48b9-81ea-254d34409f64 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:36.498261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:36.499168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.499443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.499760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.519689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:36.528860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:36.529796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:36.530630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 166/337] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:36.537491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:36.654698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445ba1-6eb4-459b-807a-53234e08d7f2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:36.701751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:36.702347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.702625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.702920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.703679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:36.703981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:36.705366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9766b9-2f48-4172-9481-255c7b764248 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:36.714732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 172/338] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:36.720453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:36.801590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:36.839684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a1b8-25e1-4526-9064-967b8d2f7d64 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:36.858729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea778d-50d0-4c00-9c9e-b81f5d212027 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.859038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea778d-50d0-4c00-9c9e-b81f5d212027 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.859469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea778d-50d0-4c00-9c9e-b81f5d212027 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.860980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 173/339] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:36.866643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.867132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.867560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.876729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:36.877380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:36.877661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:36.877932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:36.887827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:36.889225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:36.986814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:36.992405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:37.003106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.025534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 174/340] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:37.062805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.109536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.171185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 167/341] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:37.176614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.246590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a4eaa-abee-4f75-b85c-0e318abcebdd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.305496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.307261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.311235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.311745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.330943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3a2f0e3b-d5c3-411f-bf3e-1d40ab82d2f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:37.339160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:37.340166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:37.341137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:37.343520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 175/342] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:37.344922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f4bf-0f2b-4f25-a273-417abd12763a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.345344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f4bf-0f2b-4f25-a273-417abd12763a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.345762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f4bf-0f2b-4f25-a273-417abd12763a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.347228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 176/343] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:37.356257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92f985-6170-4b28-9f66-1ae7134ab994 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.356754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92f985-6170-4b28-9f66-1ae7134ab994 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.357178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92f985-6170-4b28-9f66-1ae7134ab994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.377582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d105-bd04-4038-833e-2bf3f9d37071 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.379409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cc92f985-6170-4b28-9f66-1ae7134ab994 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:37.380749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc92f985-6170-4b28-9f66-1ae7134ab994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:37.433307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.435239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.437482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.437482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.437482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.438252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:37.439726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:37.442549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 168/344] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:37.443487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.456090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 177/345] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:37.457541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021e5e-ea12-44be-8033-639f7de7b5ed None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.457826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021e5e-ea12-44be-8033-639f7de7b5ed None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.458094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021e5e-ea12-44be-8033-639f7de7b5ed None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.459571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 178/346] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:37.460252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021e5e-ea12-44be-8033-639f7de7b5ed None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.504533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.575611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2622d1-7484-4d47-9573-40da64a490ff None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.580285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021e5e-ea12-44be-8033-639f7de7b5ed None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.625299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.626197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.626367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.626548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.627342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.627748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.627883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.628046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.628802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:37.630014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:37.631922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:37.635289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:37.636741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cce5e-8283-467c-a24a-66f13c80f3b8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:37.638719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 179/347] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:37.640097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.640576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.641028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.648400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 169/348] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:37.649301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.660007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:37.661442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:37.730494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.757909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 180/349] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:37.758758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.799988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:37.806156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fbe63-1de4-4271-92c4-87208f562798 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.855561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.856786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.857326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.857834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.870046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:37.874419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785971b6-7e9b-4e4c-92be-1fe2a935b2e3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:37.880481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:37.881601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:37.882612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:37.887481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 170/350] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:37.888552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:37.917819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:37.918770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:37.919273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:37.919722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:37.920740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=23fdf0097d574eceb9554720e660ec1c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:37.921203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:37.922869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:37.929501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 181/351] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=87cd2d147f344fa99848788e1a735d8f => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:37.930555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.001265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.021364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb71e6a-939c-4819-aac7-74787ceaf244 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.064574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.065443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.065733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.066134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.104669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.139576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08ce1f-7f72-4f4d-a88c-e2dcee184c8b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.154914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.156429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646ed1a-d820-4988-b846-16a91b5d46d5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.182439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.183314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.183707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.183997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.200146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:38.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:38.209653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:38.210634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 182/352] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:38.211373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.299833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 171/353] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:38.300617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f819-cacf-43b7-b571-fbb186d9fedb None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.300903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f819-cacf-43b7-b571-fbb186d9fedb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.301173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7f819-cacf-43b7-b571-fbb186d9fedb None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.302569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 172/354] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:38.305573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7f819-cacf-43b7-b571-fbb186d9fedb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.335309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.343717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a4d55-dc5f-4ce1-ab32-9c2a459207c8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.374623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.391545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.392704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.393190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.393655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.394725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:38.395248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.396997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.408852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 183/355] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:38.410100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.418634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7f819-cacf-43b7-b571-fbb186d9fedb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.454075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.454740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.455053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.455325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.456076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:38.456380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.457764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe46b-f406-498e-afaf-5fe750d92a42 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.466592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 173/356] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:38.472015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.472302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.472603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.494707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:38.495956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:38.532950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d4dfd-4ca1-433b-b970-2add26670518 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.567319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 174/357] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:38.568045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.573289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.574391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.574932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.575398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.577024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.578827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.580937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 184/358] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:38.581912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.617114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.622689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.672991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c8a2b-680a-4079-ad57-6f2ce06fdcbf None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.714016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dac227-2470-4186-af04-919ff2b85505 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.717873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.720337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.720812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.754763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.755658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.755957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.756245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] PATH_INFO: `/v3/users/6ca9717358f74a95b28ee2aec7ff64db` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.760062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] RBAC: Authorizing `identity:get_user(user_id=6ca9717358f74a95b28ee2aec7ff64db)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.761599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.763510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 185/359] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/users/6ca9717358f74a95b28ee2aec7ff64db => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:38.764636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.803482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.805169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.868009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.890931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:38.894511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057cac-9b24-451c-b40c-2f9f1e018650 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:38.929281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 175/360] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:38.930135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:38.937940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:38.939256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:38.939703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:38.940137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:38.941519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:38.943142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:38.944945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 186/361] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:38.945915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.063933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8260e9c-5bac-4388-ba03-3594559f2f3c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.080502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219895b-7dbf-4f3c-84c7-52bd9a1b883d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.113683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.114491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.114770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.115070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.120610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.121163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.121438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.121712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.122673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] RBAC: Adding query filter params (role.id=205663cf9d3242d4a020abca619acbee, user.id=6ca9717358f74a95b28ee2aec7ff64db, scope.system=all) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:39.123307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.124588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.125814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-5cfb10c5-b86f-4f36-aae6-8460561e1c17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:39.133872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:39.135366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:39.136167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 176/362] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:39.137338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce297ad-ef3e-44da-a4e8-b34b6a2508cb None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.137501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce297ad-ef3e-44da-a4e8-b34b6a2508cb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.137652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce297ad-ef3e-44da-a4e8-b34b6a2508cb None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.137788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 187/363] 10.222.0.33 () {56 vars in 1308 bytes} [Wed Jun 15 00:21:38 2022] GET /identity/v3/role_assignments?user.id=6ca9717358f74a95b28ee2aec7ff64db&scope.system=all&role.id=205663cf9d3242d4a020abca619acbee => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.138492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.143795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 177/364] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:39.144710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce297ad-ef3e-44da-a4e8-b34b6a2508cb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.163095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.171346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.289913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f3ff-43eb-45a1-9745-28faa9bac495 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.307713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce297ad-ef3e-44da-a4e8-b34b6a2508cb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.349306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.350331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.350642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.351030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.353775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.355313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.357059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 188/365] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.357824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.362983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.364629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.364913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.365187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.366037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=23fdf0097d574eceb9554720e660ec1c) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:39.366346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.367874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752c4fc-40a0-473f-8f05-1dc3cb47c4be None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.376857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 178/366] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.378722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.379326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.379805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.400852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:39.402295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:39.480324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 179/367] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:39.481529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.482554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.490895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.495136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a1b86-b7e8-4313-bc36-67208caaec84 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.537179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.538232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.538727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.539227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.540878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:39.541405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.543110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.553014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 189/368] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.554302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.598058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383ead6-e9a6-4fcd-b47a-d2f76456e88a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.638897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.640811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.641103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.641383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.643600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.645244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.648528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 180/369] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.650095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.686896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c561ee-f4d7-4cb2-b52e-11268b9f449c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.732259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.733323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.733765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.734453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.745316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.752544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:39.763024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:39.764120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:39.765126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:39.768445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 190/370] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:39.769656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:39.776793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef73a5-3ced-4ec0-b663-6f3594e116e8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.794719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:39.823876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.824691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.824968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.825242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.908990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.909775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353c603-afbf-41ce-87c7-5b2771042492 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:39.911001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.951744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:39.952582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:39.952850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:39.953124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:39.953957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=9283a59b6fdf429b88bad064de7440f8) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:39.954267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:39.955750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:39.964956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 191/371] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:39.971826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.000576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.040894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 181/372] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:40.046540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.109386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.142806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d224ae-40a8-4314-8a7a-d81c3fb94fb1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.162856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a374273-c9b6-4a11-89cd-472286432a1b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.205280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:40.205974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.206269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.206544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.208086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:40.210540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:40.213553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 182/373] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:40.219130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.219925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:40.220833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.221292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.221768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.223099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:40.223803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:40.226182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:40.239197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 192/374] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:40.254872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.288034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.350950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99036758-a8ba-42f4-a108-4bc5718b4c93 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.392234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:40.393097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.393381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.393656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.395180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:40.397007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:40.400619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 183/375] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:40.406970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.459176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab0a6-38fb-4b94-a853-c6b354181087 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.529557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9d2e-b0c8-4625-ac3c-3a64cba68376 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.530527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:40.531463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.531851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.532318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.543899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.553113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.574703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:40.575398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.575682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.575957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.577321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=9283a59b6fdf429b88bad064de7440f8) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:40.577646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:40.579142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4fd5b-a306-4fe5-bc28-75273183de42 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:40.585811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 184/376] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=87cd2d147f344fa99848788e1a735d8f => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:40.613225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d7d7-c444-497e-ade1-97a6e68c9e9d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.613528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d7d7-c444-497e-ade1-97a6e68c9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.613795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d7d7-c444-497e-ade1-97a6e68c9e9d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.615619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 185/377] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:21:40.623359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:40.623716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:40.623925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:40.663785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:40.665694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:40.678473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:40.679236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:40.795260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 186/378] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:40.814479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.862743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 193/379] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:40.871786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:40.941957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:40.945978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e72bc-ce53-477f-8eeb-1f861690f31c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:40.950170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:41.022513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.023492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.023779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.024052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.047945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:41.062755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:41.063713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:41.064495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 187/380] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:41.065142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f275-ac3d-48fa-a762-5ddcd6b291b8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.072115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.107175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.108047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.108331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.108608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] PATH_INFO: `/v3/projects/bc3f10f7de5140a991c9148b14dd9e73` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.110577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] RBAC: Authorizing `identity:get_project(project_id=bc3f10f7de5140a991c9148b14dd9e73)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.112070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.113786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 194/381] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:40 2022] GET /identity/v3/projects/bc3f10f7de5140a991c9148b14dd9e73 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:41.123845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.262529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853caa-b885-43d7-8eb6-f27cb5714391 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.273479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:41.308093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b901f6-a9f4-4270-9578-7ffa56ca53ab None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.309167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.310538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.311045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.311532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] PATH_INFO: `/v3/groups/fbe3b9aac67749da87d1b972029aa6f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.320360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] RBAC: Authorizing `identity:get_group(group_id=fbe3b9aac67749da87d1b972029aa6f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.322641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.325494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 195/382] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:41 2022] GET /identity/v3/groups/fbe3b9aac67749da87d1b972029aa6f8 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:41.331723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.398361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.399081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.399673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.400140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.401278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:41.401395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.402760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.419087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 188/383] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:41.423570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.456852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9efd-e7a1-4cb4-9697-debd25585252 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.488366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:41.512984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.514336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.514861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.515842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.516885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] RBAC: Adding query filter params (group.id=fbe3b9aac67749da87d1b972029aa6f8, role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=bc3f10f7de5140a991c9148b14dd9e73) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:41.517396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.519063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.527210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 196/384] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:41 2022] GET /identity/v3/role_assignments?group.id=fbe3b9aac67749da87d1b972029aa6f8&scope.project.id=bc3f10f7de5140a991c9148b14dd9e73&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:41.621586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a41638-797e-4996-bbda-429296592ae8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.661578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.662450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.662732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.663029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.664262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.666327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.668399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 189/385] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:41 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:41.674092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.733017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:41.810681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15959fdd-7dc0-4636-946e-65b78c80fbea None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:41.852687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:41.854218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:41.854731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:41.855295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:41.856765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:41.858399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520287f-d936-43aa-a854-deb5717e9332 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:41.860645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 197/386] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:41 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:41.865917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:41.993171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:42.000557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfdb26-ea64-4ef1-b263-e5bc47ee18da None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:42.044994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:42.046213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.046692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.049862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.052498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, group_id=9667a32e2e70471e99a37c4bfdc146f3, role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:42.054147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5138e8d-4619-4c31-93d0-5835ad7ee10d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:42.079860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 190/387] 10.222.0.33 () {58 vars in 1442 bytes} [Wed Jun 15 00:21:41 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/87cd2d147f344fa99848788e1a735d8f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:42.291278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37444-f7f7-4245-83a6-a15ff63c8af6 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.291948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37444-f7f7-4245-83a6-a15ff63c8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.292462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37444-f7f7-4245-83a6-a15ff63c8af6 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.294395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 198/388] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:42.300915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.301207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.301466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.330852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:42.332063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:42.412496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 191/389] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:42.431824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37444-f7f7-4245-83a6-a15ff63c8af6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:42.551838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a37444-f7f7-4245-83a6-a15ff63c8af6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:42.554224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:42.592918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:42.593713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.593985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.594252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.609126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:42.616944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:42.617828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:42.619274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 199/390] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:42.624930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:42.726524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:42.747148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ccd3-fa04-41bb-84d1-7d7f36dea0ad None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:42.785809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:42.786469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.786747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.787044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.787793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:42.788091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:42.789453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:42.802962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 192/391] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:42.804472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:42.916962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc15d0b-c5f6-420b-96d4-e97924e7312e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:42.957855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:42.958729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:42.959041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:42.959320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:42.963468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:42.964940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:42.966901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 200/392] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:42 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:42.972250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.118607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b29fa3-8d6c-492e-b7e3-d55f4c504761 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.168364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.169660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.170142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.170610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.172203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:43.173905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4a98d-f618-4f4b-bd32-1d5e87a9326b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:43.176048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 193/393] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:42 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:43.182137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.235273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.305782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4fe78-8d9e-4876-b647-4aaefbde7b1b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.306396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4fe78-8d9e-4876-b647-4aaefbde7b1b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.306934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4fe78-8d9e-4876-b647-4aaefbde7b1b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.308502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 194/394] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:43.311910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb87c30-bfab-4d52-80cc-7448aef1386e None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.312195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb87c30-bfab-4d52-80cc-7448aef1386e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.312469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb87c30-bfab-4d52-80cc-7448aef1386e None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.316239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 195/395] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:43.317692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34bf3-bc0f-403b-9327-a52566649e0b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.317973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34bf3-bc0f-403b-9327-a52566649e0b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.318246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34bf3-bc0f-403b-9327-a52566649e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.335982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6b961-4aa7-4696-9e13-6b5b09134a9c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.336287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-23d34bf3-bc0f-403b-9327-a52566649e0b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:43.339768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-23d34bf3-bc0f-403b-9327-a52566649e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:43.349842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.397767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.399548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.400506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.400949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.403590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] RBAC: Authorizing `identity:create_grant(project_id=23fdf0097d574eceb9554720e660ec1c, group_id=9667a32e2e70471e99a37c4bfdc146f3, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:43.406006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:43.425342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 196/396] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:43.427374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.427862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.428294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.431732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 201/397] 10.222.0.33 () {58 vars in 1442 bytes} [Wed Jun 15 00:21:43 2022] PUT /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:43.448787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:43.450447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:43.483131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.554534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 197/398] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:43.578019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.610192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.620560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf5e5-f935-4c69-8d6a-4786e6c5ead1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.658230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.659090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.659436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.659776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.663227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.672192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:43.681373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:43.682336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:43.683162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:43.686945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 202/399] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:43.690051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.696988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f10a-a1fd-4fbe-9695-3ce5f1bb1819 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.733564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.734885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.735412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.735924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.748471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:43.759527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:43.760468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:43.761265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:43.764051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 198/400] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:43.770288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.837744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68d295-e6dd-4612-85d4-3eead85965f1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.875466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.876152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.876431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.876681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.877429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:43.877730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:43.879230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:43.882527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cda77-efde-40b5-b80a-1dcf46525427 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:43.893278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 203/401] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:43.894674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:43.920344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:43.921338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:43.921759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:43.922187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:43.922323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.922997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:43.923309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:43.924713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:43.926366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:43.935294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 199/402] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:43.940837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.008026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd390-d712-47c5-8306-4c2c2ad14499 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.045771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d7f5-3c1b-4cc0-9a7e-222f4dba0ed6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.051657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.052808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.053282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.053759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.059615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.061479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.064703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 204/403] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:44.069843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.087238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.087238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.087238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.087238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.087608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.087608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.088003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 200/404] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:43 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:44.093385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.157406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:44.213308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4eaae-75c4-4835-8c1f-6ec652daade9 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.279224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:44.289072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.291263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.291567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.291855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.293335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.294859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.296945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 201/405] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:44 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:44.302661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.332537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a60e1-b0a7-4aef-a09e-3413d0768b75 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.372769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.373643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.373924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.374199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.375407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.376856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2e653-343c-4107-9703-64b6fdced3ce None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.378881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 205/406] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:44 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:44.380075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc735e6-e184-4392-bbda-ef12a548cf8a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.380357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc735e6-e184-4392-bbda-ef12a548cf8a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.380632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc735e6-e184-4392-bbda-ef12a548cf8a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.381853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 206/407] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:44.386738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc735e6-e184-4392-bbda-ef12a548cf8a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.425873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ec6cd-d3ed-4c39-88c8-eb7e9bf60a2f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.470696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.471941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.472506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.472795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.475197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:44.481104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, group_id=9667a32e2e70471e99a37c4bfdc146f3, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.482561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd90203-ee0f-4944-af46-b119de7ac265 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.510962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 202/408] 10.222.0.33 () {58 vars in 1442 bytes} [Wed Jun 15 00:21:44 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:44.510962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.510962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.510962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.511714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc735e6-e184-4392-bbda-ef12a548cf8a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.550804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:44.555029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:44.589740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.590636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.590965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.592195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.598936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] RBAC: Authorizing `identity:create_grant(project_id=9283a59b6fdf429b88bad064de7440f8, group_id=9667a32e2e70471e99a37c4bfdc146f3, role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:44.602344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:44.626756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 207/409] 10.222.0.33 () {58 vars in 1442 bytes} [Wed Jun 15 00:21:44 2022] PUT /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8/groups/9667a32e2e70471e99a37c4bfdc146f3/roles/87cd2d147f344fa99848788e1a735d8f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:44.689299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 203/410] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:44.724740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.739174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:44.849995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea07040-c936-4ead-85c5-842d324c98af None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:44.898295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:44.900123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.900637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.901107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.914152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5ba252ff-6ed5-4c2b-afd0-60901108c70a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:44.922764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:44.923980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:44.924772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:44.927600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 208/411] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:44.934485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:44.960254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0726-84ed-4ea5-ba09-7bec9ce6216d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.960888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0726-84ed-4ea5-ba09-7bec9ce6216d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.961362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0726-84ed-4ea5-ba09-7bec9ce6216d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.963609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 209/412] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:44.971887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:44.972798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:44.973583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:44.993300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:44.995048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:45.070579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:45.077364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 210/413] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:45.111027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.174343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e20bbc-629b-4924-8bf9-ea98d0ddad96 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.206138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:45.224808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.226476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.228758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.229046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.231440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:45.232029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:45.234490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:45.254087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 204/414] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:45.264331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.305317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9179-0ee8-45a2-a852-9266280c782a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.350239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.351389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.351676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.351980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.370821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:45.379253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:45.380148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:45.380947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 211/415] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:45.386532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.453160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbda5e4-620c-4cd1-8f47-ea3fd6c21353 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.459479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:45.530536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286636b-31ea-461a-90fa-2e2416dd884a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.536050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.536912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.537191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.537462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] PATH_INFO: `/v3/groups/fbe3b9aac67749da87d1b972029aa6f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.543552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] RBAC: Authorizing `identity:get_group(group_id=fbe3b9aac67749da87d1b972029aa6f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:45.545016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:45.546677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 205/416] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:45 2022] GET /identity/v3/groups/fbe3b9aac67749da87d1b972029aa6f8 => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:45.552627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.602064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.602766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.603335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.603610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.604368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:45.604680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:45.606059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:45.631469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 212/417] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:45.637526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.663307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:45.761222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cb04d-6217-4a65-948c-d7a12c4297f2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.788382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:45.792986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309b83-163c-439f-b700-fe3f9a94bf19 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:45.803177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.803837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.804121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.804420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.876966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:45.877836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:45.878116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:45.878403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] PATH_INFO: `/v3/projects/bc3f10f7de5140a991c9148b14dd9e73` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:45.881708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] RBAC: Authorizing `identity:get_project(project_id=bc3f10f7de5140a991c9148b14dd9e73)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:45.883198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:45.886591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 206/418] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:45 2022] GET /identity/v3/projects/bc3f10f7de5140a991c9148b14dd9e73 => generated 344 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:45.901034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:45.902347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:45.906297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.080398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb37f-b703-410c-823e-451f71c31415 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.107266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:46.123352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.124247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.124529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.124820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] PATH_INFO: `/v3/projects/bc3f10f7de5140a991c9148b14dd9e73/groups/fbe3b9aac67749da87d1b972029aa6f8/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.132222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc3f10f7de5140a991c9148b14dd9e73, group_id=fbe3b9aac67749da87d1b972029aa6f8, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:46.133689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff29381-ce1f-4246-8e0d-8b6ecad90c66 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.141172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 213/419] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 00:21:46.148390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.162637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 207/420] 10.222.0.33 () {58 vars in 1442 bytes} [Wed Jun 15 00:21:45 2022] PUT /identity/v3/projects/bc3f10f7de5140a991c9148b14dd9e73/groups/fbe3b9aac67749da87d1b972029aa6f8/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:21:46.163982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606763a-e609-45b5-a1d0-fe98ac6f84f8 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.164856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606763a-e609-45b5-a1d0-fe98ac6f84f8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.165273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606763a-e609-45b5-a1d0-fe98ac6f84f8 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.167239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 208/421] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:46.176494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.176952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.177270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.197964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:46.199334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:46.212837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:46.283694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 209/422] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:46.320128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.394049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff77e4d-e689-48f8-994d-eb173619751a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.432546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.433398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.433682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.434832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.436068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:46.437510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.439563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 214/423] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:46.447703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.466667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9bac-c475-447e-9eb1-58b0a9de41d6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.481633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:46.547369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.548183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.548469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.548746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.551165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:46.560598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b69ae-4d3f-4d23-908a-89965db6caf2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.587905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:46.595735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:46.596633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:46.597428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 210/424] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:46.604226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.605098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.605378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.605651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.606887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:46.607738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.608348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.610335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 215/425] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:46.616047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.768717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4abca-f031-484b-8915-58950e2525a4 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.805693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643dafbd-d1b4-4dd6-81f4-06568ad73577 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.809557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.810286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.810572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.810882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.812515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:46.812836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:46.814217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.825006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 211/426] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:46.830373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:46.866186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:46.884111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.884775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.885055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.885327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.886166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=23fdf0097d574eceb9554720e660ec1c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:46.886473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:46.887939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daea3d-6d59-49d5-a082-9bbea2777a38 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:46.913450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 216/427] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=87cd2d147f344fa99848788e1a735d8f => generated 611 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:46.954387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703c9d3-ed2e-4eba-a4f1-c5d502ee6734 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:46.991932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:46.992595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.992875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.993873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.996404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ddfd-b62b-46cc-a480-f3c79de636a7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:46.996693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ddfd-b62b-46cc-a480-f3c79de636a7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:46.996971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ddfd-b62b-46cc-a480-f3c79de636a7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:46.998535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 217/428] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:47.008299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.008592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.008914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.054924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:47.056639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:47.073093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.074582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.117234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.125449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.187382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 218/429] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:47.201252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 212/430] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 00:21:47.202225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.211457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.315753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224476c3-2b93-4c52-b87b-8f9a191333c6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.335292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56870e-9c11-46ee-b14e-9b95892fbe22 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.353776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.354638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.354945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.355219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] PATH_INFO: `/v3/projects/23fdf0097d574eceb9554720e660ec1c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.356434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] RBAC: Authorizing `identity:get_project(project_id=23fdf0097d574eceb9554720e660ec1c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.357896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a2b8e-0863-48bd-aed7-0d9c95abd32d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.359948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 219/431] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/projects/23fdf0097d574eceb9554720e660ec1c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:47.365771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c1b1d-44d1-4ea4-96e8-e871370f6b5e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.366055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c1b1d-44d1-4ea4-96e8-e871370f6b5e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.366333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c1b1d-44d1-4ea4-96e8-e871370f6b5e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.367900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 220/432] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:47.368636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c1b1d-44d1-4ea4-96e8-e871370f6b5e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.374903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.382430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.383774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.384072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.384470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.395241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.395564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-47f5bfbd-f82a-49b3-924d-297bc55550aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:47.404437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:47.405394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:47.406192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 213/433] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:47.410909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.411593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.411874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.430433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:47.431668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:47.489865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c1b1d-44d1-4ea4-96e8-e871370f6b5e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.503176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 214/434] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:47.515622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.527303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.528157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.528439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.528770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.529961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.531497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.533190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 221/435] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:47.536197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.633156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.633697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47ab4-08bc-4b21-a102-5bc7846d7854 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.650541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245432d1-8e8b-4f46-8f37-20d1a36118cd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.663565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.676329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.676990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.677268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.677538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.678280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] RBAC: Adding query filter params (name=member) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:47.678587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.680003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.688469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 215/436] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:47.689241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.694288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.695120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.695398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.695668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.707913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:47.717571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:47.718433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:47.719280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:47.721538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 222/437] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:47.722421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.794427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8cf36-7fea-4405-96a1-6b707db7d717 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.828697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6aa992-d418-48d3-9222-99d690b15fa5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:47.830752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.832317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.832625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.832879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.833711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=23fdf0097d574eceb9554720e660ec1c) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:47.834023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.835470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.850802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 216/438] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=23fdf0097d574eceb9554720e660ec1c&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:47.850802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:47.882165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:47.883284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:47.884110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:47.884935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:47.915174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.915537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:47.975576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:47.977551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:47.981972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0c86-9c18-4d19-be51-2c0393d8c287 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.019149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.019352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.019601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.019760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.020494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:48.020800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.022185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.032003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 217/439] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:48.038008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.101192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 223/440] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:48.106544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.167479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.173854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d953c3-8926-4777-b503-b6b53c7fe2a3 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.184555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.220079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.220734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.221419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.221705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.238500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db515e16-7243-45d9-be56-416d4fafb060 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.290154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.291310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.291695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.292069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.293757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.294755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.296790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 224/441] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:48.304048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.317830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.319920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.438192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29159-c1dd-4f15-9068-e2aa4ced0522 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.458792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 218/442] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:48.458792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.463116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.471380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.488978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.489852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.490129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.490400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.494092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.498389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.498933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 225/443] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:48.505133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.587016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4a26-6c39-464b-b4c6-31ab860f03d0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.617448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d94292-1dd1-4e40-8682-b3b15b2d35aa None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.621090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.621937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.622209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.622481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] PATH_INFO: `/v3/projects/9283a59b6fdf429b88bad064de7440f8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.623670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] RBAC: Authorizing `identity:get_project(project_id=9283a59b6fdf429b88bad064de7440f8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.625093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1692-ab0f-419a-8f25-69c5f9371158 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.626767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 219/444] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/projects/9283a59b6fdf429b88bad064de7440f8 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:48.627923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab9a62-a210-41d6-a0a7-7ff10cc3c6b8 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.628202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab9a62-a210-41d6-a0a7-7ff10cc3c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.628471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab9a62-a210-41d6-a0a7-7ff10cc3c6b8 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.629793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 220/445] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:48.632111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab9a62-a210-41d6-a0a7-7ff10cc3c6b8 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.652947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.653593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.653867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.654144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.655003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=b9e44997cbef45b98a11c68cadd4cbf3, scope.project.id=9283a59b6fdf429b88bad064de7440f8) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:48.655313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.656726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aa2c2-f49c-4b39-9e77-d4396d924143 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.662436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 226/446] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=b9e44997cbef45b98a11c68cadd4cbf3 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:48.663731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.664015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.664285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.682266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:48.683546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:48.732895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.734912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:48.761590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 227/447] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:48.763969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab9a62-a210-41d6-a0a7-7ff10cc3c6b8 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.784565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.800883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.801745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.802020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.802288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] PATH_INFO: `/v3/groups/9667a32e2e70471e99a37c4bfdc146f3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.803561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] RBAC: Authorizing `identity:get_group(group_id=9667a32e2e70471e99a37c4bfdc146f3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.805075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.807423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 221/448] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/groups/9667a32e2e70471e99a37c4bfdc146f3 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:48.813857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:48.908605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebfb76-fa2d-4af4-bff8-793d82c2d19a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.924055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83f59-f399-47ae-99bf-c9062249b10e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:48.953526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.954318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.954595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.954898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.960854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:48.961501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:48.962382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:48.962652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:48.963630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] RBAC: Adding query filter params (group.id=9667a32e2e70471e99a37c4bfdc146f3, role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=9283a59b6fdf429b88bad064de7440f8) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:48.963879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:48.965375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:48.970820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:48.972001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 222/449] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/role_assignments?group.id=9667a32e2e70471e99a37c4bfdc146f3&scope.project.id=9283a59b6fdf429b88bad064de7440f8&role.id=87cd2d147f344fa99848788e1a735d8f => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:48.979149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:48.980088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:48.980883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 228/450] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:48.986549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:49.001786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:49.098877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107d6af-126b-4bfb-b399-60f6f923e6c1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:49.133319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:49.134016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:49.134294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:49.134564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:49.135343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:49.135647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:49.137009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:49.145105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 223/451] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:49.151171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:49.192271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:49.256726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787bcf6-fa2a-4fa3-bc1f-3c12315ac60b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:49.292271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:49.292925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:49.293203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:49.293473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:49.360529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:49.361989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:49.398919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:49.463892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 229/452] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:49.469497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:49.537856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:49.566142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1e647-d96c-4bcd-acd4-98a01961c81a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:49.599808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:49.600666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:49.600948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:49.601221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] PATH_INFO: `/v3/projects/bc3f10f7de5140a991c9148b14dd9e73` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:49.602452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] RBAC: Authorizing `identity:get_project(project_id=bc3f10f7de5140a991c9148b14dd9e73)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:49.604188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:49.605884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 224/453] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:21:49 2022] GET /identity/v3/projects/bc3f10f7de5140a991c9148b14dd9e73 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:49.611502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:49.712091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404a89-122c-4edd-9e90-21aa7d18d372 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:49.745736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:49.746576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:49.746882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:49.747157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] PATH_INFO: `/v3/groups/fbe3b9aac67749da87d1b972029aa6f8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:49.748322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] RBAC: Authorizing `identity:get_group(group_id=fbe3b9aac67749da87d1b972029aa6f8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:49.749761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89eb80-834a-4972-a3b7-a54e987295d5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:49.751396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 230/454] 10.222.0.33 () {56 vars in 1180 bytes} [Wed Jun 15 00:21:49 2022] GET /identity/v3/groups/fbe3b9aac67749da87d1b972029aa6f8 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:49.757649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:49.855531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589e1b-b8f7-49db-bf8c-7403a288d936 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:49.891612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:49.892375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:49.892657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:49.892927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:49.893755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] RBAC: Adding query filter params (group.id=fbe3b9aac67749da87d1b972029aa6f8, role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=bc3f10f7de5140a991c9148b14dd9e73) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:49.894064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:49.895510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e63f4-e54c-4dee-b1a6-c42b38fe21c0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:49.901303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 225/455] 10.222.0.33 () {56 vars in 1376 bytes} [Wed Jun 15 00:21:49 2022] GET /identity/v3/role_assignments?group.id=fbe3b9aac67749da87d1b972029aa6f8&scope.project.id=bc3f10f7de5140a991c9148b14dd9e73&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:52.611396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e132b8-1b94-409c-b3a3-0cf618c807b3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.612397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e132b8-1b94-409c-b3a3-0cf618c807b3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.612907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e132b8-1b94-409c-b3a3-0cf618c807b3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:52.614801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 231/456] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:52.620392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dc929-05a2-47f6-8a54-ab338b24d015 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.620807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dc929-05a2-47f6-8a54-ab338b24d015 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.620935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dc929-05a2-47f6-8a54-ab338b24d015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:52.629487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:52.647844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-072dc929-05a2-47f6-8a54-ab338b24d015 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:52.649425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-072dc929-05a2-47f6-8a54-ab338b24d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:52.741768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 226/457] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:52.767340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e132b8-1b94-409c-b3a3-0cf618c807b3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:52.777445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:52.813809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bad77a-2655-4f29-b736-f5c13811d592 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.814149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bad77a-2655-4f29-b736-f5c13811d592 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.814505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bad77a-2655-4f29-b736-f5c13811d592 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:52.816023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 227/458] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:52.821844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517821-7dd3-4966-b083-67de5d3fdb0f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.822393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517821-7dd3-4966-b083-67de5d3fdb0f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.822891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517821-7dd3-4966-b083-67de5d3fdb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:52.853892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-fc517821-7dd3-4966-b083-67de5d3fdb0f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:52.856359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-fc517821-7dd3-4966-b083-67de5d3fdb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:52.941098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e132b8-1b94-409c-b3a3-0cf618c807b3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:52.962626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 228/459] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:52.964275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1b13-e3ef-486d-bc3f-a448917cf141 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.964892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1b13-e3ef-486d-bc3f-a448917cf141 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.965507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1b13-e3ef-486d-bc3f-a448917cf141 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:52.968097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 229/460] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:52.977001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:52.978476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:52.979217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.001204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.001835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:53.001985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.002426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.002882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.003602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:53.016936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.018840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:53.044898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:53.045816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:53.046670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:53.046670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:53.046670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.046670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:53.046670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 232/461] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:53.047603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.072284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.090842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 230/462] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:53.091932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.121988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.178725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2fb5a-ba3e-43ad-bb8c-5fa69ba0b9a6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.216861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f826b-31ea-43de-88bf-364a2b7d6769 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.223588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.224519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.224995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.225363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.245411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.247857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.265896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.266923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.267351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.267821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.268711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:53.269248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.270889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:53.271844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:53.272869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:53.273849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:53.273849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:53.273849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.273849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:53.280042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 233/463] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:53.281763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.295923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 231/464] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:53.297768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.364011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.411603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d33a9-25eb-4bc7-aa3e-35233df64164 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.414496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf734-71de-4c6e-b049-1aad7dfe090e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.424100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.449654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.450314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.450593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.450690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.450890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.451479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.451589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:53.451751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.451900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.452045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.453262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.464678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.470200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 232/465] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:53.470340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.470431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2a4281fa-0282-4365-9f39-5edcf0f9334f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:53.491764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:53.492692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:53.494215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:53.494215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:53.494215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:53.494215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:53.494215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 234/466] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:53.494606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085afc-2bd8-48b2-a7cb-c7a41a1d7f08 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.494875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085afc-2bd8-48b2-a7cb-c7a41a1d7f08 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.495155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f085afc-2bd8-48b2-a7cb-c7a41a1d7f08 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.496453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 235/467] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:53.501067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f085afc-2bd8-48b2-a7cb-c7a41a1d7f08 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.577837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0214a2-3c4d-43d5-96e7-adfa0958dfe9 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.593118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.610721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f085afc-2bd8-48b2-a7cb-c7a41a1d7f08 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.616068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.616757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.617039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.617313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.618090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.619645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.652161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.652949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.653311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.653661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.654408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.655208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 233/468] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:53 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:53.655943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103d4d7-bce5-42c6-81ec-3f9256850f6c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.656058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.683639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 236/469] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:53 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:53.684959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1163-fdeb-4473-a8d5-73070874c902 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.685257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1163-fdeb-4473-a8d5-73070874c902 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.685542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e1163-fdeb-4473-a8d5-73070874c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.708704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fe2e1163-fdeb-4473-a8d5-73070874c902 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:53.710092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fe2e1163-fdeb-4473-a8d5-73070874c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:53.736719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.800575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea39a39-bae1-43e6-8433-dd1431314e04 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:53.824030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 237/470] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:53.825793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084162c7-5579-45b9-b1c1-9aa9e16dc549 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.826288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084162c7-5579-45b9-b1c1-9aa9e16dc549 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.826726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084162c7-5579-45b9-b1c1-9aa9e16dc549 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.828800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 238/471] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:53.834881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.835415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.836158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.849674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:53.850690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:53.851005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:53.851282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:53.852555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:53.853124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:53.854534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:53.859652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:53.860962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:53.871380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 234/472] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:53.872450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.909498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:53.959685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 239/473] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:53.960523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:53.993718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb36d-3abf-44ac-84d6-7d001492727a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.034540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.035646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.035936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.036214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.052548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.054022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.063260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:54.078166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:54.079144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 235/474] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:54.080415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:54.087616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be1a8b-078d-4a6f-8595-d6c9950a9386 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.126074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.127142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.127625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.128094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.129053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.130871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.155275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 240/475] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:53 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:54.156528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:54.206370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:54.235310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0e2f5-5f32-498a-94c8-f4675492357c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.292062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c374616-ce25-4666-b1bc-b3a8bdbcb76e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.322811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.324027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.324508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.324981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.327051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.328867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.338942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.sql.core [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 00:21:54.338942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 00:21:54.338942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [parameters: {'id': 'b9f885044d8b4bff85257cd86606884e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 00:21:54.338942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80266) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application result.read() Jun 15 00:21:54.359315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:21:54.360227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application yield resource Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise self.value Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application session.commit() Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 00:21:54.361169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self.session.flush() Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application compat.raise_( Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 00:21:54.362025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:21:54.362977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application result.read() Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:21:54.365720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [parameters: {'id': 'b9f885044d8b4bff85257cd86606884e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:54.366698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 00:21:54.369470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.371116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 236/476] 10.222.0.33 () {60 vars in 1131 bytes} [Wed Jun 15 00:21:54 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:54.374587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:54.401651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.402666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.403216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.404206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.405421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:54.405908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.408177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.410964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:54.427645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 241/477] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:54.432814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:54.506480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ae14-6d13-4f9b-80a3-2b4c7a0b9929 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.543504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.544091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.544381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.544653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.558018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:54.559877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a5d4-ec36-4bcc-9ad5-6af2d093b7a9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:54.566241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:54.567173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:54.568173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 237/478] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:21:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:21:54.573176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:54.583375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:54.599258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.600236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.600672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.601084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.602533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.604469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06eaf2-e8fd-42b4-b8d1-06a44beb7377 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.627901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 242/479] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:54 2022] POST /identity/v3/users => generated 249 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:21:54.694603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f631e-8c75-4374-920a-376a02acb188 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:54.735218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:54.735945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:54.736283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:54.736569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:54.737319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:54.737628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:54.739109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9869e5c-294c-4624-807b-62ccedd081e6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:54.749799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 238/480] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:56.147643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612ce2d-b661-475c-a2c0-3c7f69272e4b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.148168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612ce2d-b661-475c-a2c0-3c7f69272e4b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.148168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612ce2d-b661-475c-a2c0-3c7f69272e4b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.150580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 243/481] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:56.155689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e8e5-141e-4e8e-9ecb-3bce50b2117d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.156087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e8e5-141e-4e8e-9ecb-3bce50b2117d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.156265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e8e5-141e-4e8e-9ecb-3bce50b2117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.182559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9739e8e5-141e-4e8e-9ecb-3bce50b2117d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:56.183784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9739e8e5-141e-4e8e-9ecb-3bce50b2117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:56.263753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:56.273338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 239/482] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:56.307661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612ce2d-b661-475c-a2c0-3c7f69272e4b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:56.394547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:56.441120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837fb9-f343-4f5d-8265-ef14b51934bd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.441400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837fb9-f343-4f5d-8265-ef14b51934bd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.441602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837fb9-f343-4f5d-8265-ef14b51934bd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.443259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 240/483] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:56.449577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.449871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.450153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.461709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612ce2d-b661-475c-a2c0-3c7f69272e4b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:56.486698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:56.487940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:56.500997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:56.501858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.502143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.502420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.521457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:56.529270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:56.530181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:56.531031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:56.532139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 244/484] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:56.541630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:56.626886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 241/485] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:21:56.697348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:56.707194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f47e8d-4d54-4182-bdbb-d5c5bd2d26b7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:56.727398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:56.774541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:56.775239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.775526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.775795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.776554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:56.776859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:56.778230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:56.798555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 245/486] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:21:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:56.802147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:56.881333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f092cbd-0d5f-4abc-8014-756549dfe6c3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:56.934379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:56.955769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:56.956507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:56.956789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:56.957067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:56.979485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9439-94e0-4801-bf13-0fb9108fe3d1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:56.991437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:56.999863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:57.000778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:57.001558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 242/487] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:57.006949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.044609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.045405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.046867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.047148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.064310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.065812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:57.103630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:57.104560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:57.105413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:57.105413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:57.105413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:57.105413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:57.107772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 246/488] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:57.112955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.147192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.174723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392f9-b597-4d62-8334-a42fe1ac4bfa None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.213543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.214245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.214529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.214833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.215592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:57.215905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.217413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1589abb-f754-49f0-8b3e-5050cc4f5fee None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.231518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 243/489] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:57.232916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab5f2-6564-4535-9ffa-c870a9926cfb None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.233253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab5f2-6564-4535-9ffa-c870a9926cfb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.233533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab5f2-6564-4535-9ffa-c870a9926cfb None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.235104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 244/490] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:57.239627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab5f2-6564-4535-9ffa-c870a9926cfb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.241907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e6306-7c5c-4939-9319-788f43084703 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.279646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.280696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.281157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.281616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.282597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:57.283187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.284814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068648b0-639a-4870-b4c8-13a1b5bcf3c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.295116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.299146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 247/491] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:21:57.300816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.301318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.301769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.324328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:57.326018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:57.358919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab5f2-6564-4535-9ffa-c870a9926cfb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.394267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.395119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.395793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.396099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.400850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 248/492] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:57.402032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.416217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.417752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.426407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-88798841-1214-4772-b448-cfe0b743f276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:57.439106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:57.440054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:57.440900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:57.440900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:57.440900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:57.440900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:57.440900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 245/493] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:57.441542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d0b-fdaf-4716-aa47-620b0e224579 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.441823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d0b-fdaf-4716-aa47-620b0e224579 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.442094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd5d0b-fdaf-4716-aa47-620b0e224579 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.443627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 246/494] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:57.444319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd5d0b-fdaf-4716-aa47-620b0e224579 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.519947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52464018-0932-4e2b-91db-12e0f85e803d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.553878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.554668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.555546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.555799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.557990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd5d0b-fdaf-4716-aa47-620b0e224579 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.560064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.563197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.564662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2a87a6b3-6ee5-4afd-8d05-ed9ed2bb8c49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:21:57.580264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:57.581328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:57.582372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 249/495] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:57.583133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6142a-f971-406f-926f-9e78c4283471 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.583296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6142a-f971-406f-926f-9e78c4283471 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.583580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6142a-f971-406f-926f-9e78c4283471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.594353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.595187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.595468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.595742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.599689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9ab6142a-f971-406f-926f-9e78c4283471 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:57.600877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9ab6142a-f971-406f-926f-9e78c4283471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:57.606506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:57.614075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:57.615062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 247/496] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:57.616102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.669746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 250/497] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:57.671386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3523-41ec-43b3-97c0-047a4d443c12 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.671674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3523-41ec-43b3-97c0-047a4d443c12 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.671949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3523-41ec-43b3-97c0-047a4d443c12 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.673211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 251/498] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:21:57.673902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c3523-41ec-43b3-97c0-047a4d443c12 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.677568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.725643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e37e1-f20d-4067-a699-9815f3957a2a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.759165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.759748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.760030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.760302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.761020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:57.761328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.762700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.774189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 248/499] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:57.775060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.776009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c3523-41ec-43b3-97c0-047a4d443c12 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.799106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.810524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.811209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.811488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.811761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.812587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:57.812890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.814322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f1ad0-3e3e-48fd-8a1b-44ab5e11b5c6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.849643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 252/500] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:57.850865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c70dc-b442-4244-a792-55f4023297ce None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.851150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c70dc-b442-4244-a792-55f4023297ce None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.851421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c70dc-b442-4244-a792-55f4023297ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.867699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-170c70dc-b442-4244-a792-55f4023297ce None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:21:57.868886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-170c70dc-b442-4244-a792-55f4023297ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:21:57.877723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a719bd-1709-4c83-b774-a8fccc49f2ac None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:57.910743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:57.911414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:57.911691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:57.911961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:57.912694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:57.912994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:57.914957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:57.924733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:57.926958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 249/501] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:57.928012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:57.935371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 253/502] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:21:57.936160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c70dc-b442-4244-a792-55f4023297ce None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.013713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.029874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0d58-9afb-4dc2-90ee-64a7715cc44e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.032233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c70dc-b442-4244-a792-55f4023297ce None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.063434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.064276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.064559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.064892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.065490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.066268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.066536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.066836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.073621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.075086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.075215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:21:58.082878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.083893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 250/503] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.084791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:21:58.089737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.090768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 254/504] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.091780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.142374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.188099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687967-1a58-4e98-94a4-659a729bf713 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.189574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c9c07-0c5a-4dd8-beea-ebfa69db710a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.221086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.221711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.221986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.222252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.222348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.223163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.223446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.223779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.234861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.236316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.238330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:58.253403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.254748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.255902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:58.255902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:58.255902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.255902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:58.259708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 255/505] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.260368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.293030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.294408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.358677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2468b-e8de-44e2-979e-7480245f0e76 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.360957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.392488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.393274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.393556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.393832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.396150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 251/506] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:58.396974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.410135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.411629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:58.428844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.429726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.430588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:58.430588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:58.430588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.430588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:58.430588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 256/507] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.430887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.441486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.496734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5481f7-59be-4d78-b154-19f2c3b6c3ff None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.531821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed40a78-b240-403c-8bce-8b4463c458cb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.539087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.539775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.540055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.540327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.541079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:58.541389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.542787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.550910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 252/508] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:58.551713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.564757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.565415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.565692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.565959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.566804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:58.567132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.568584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.588047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 257/509] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:58.588855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.603664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.653670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ff0bf-2586-41ea-8cd3-d71a59c593af None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.660826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.687659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c450b04-fb46-4126-846b-15cbba3f616f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.689159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.689812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.690091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.690361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.691116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:58.691423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.692794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.704263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 253/510] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:58.705035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.725525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.726329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.726607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.726905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.736786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.738224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:58.755666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.756528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.757362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:58.757362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:58.757362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.757362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:58.759059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 258/511] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.760424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.801734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f873b-b3b9-4a77-965e-1ee92a847977 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.824003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.835703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.836381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.836655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.836922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.837861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:58.837965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.839296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.850469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 254/512] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:58.851260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.859298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d268-077b-49cd-a084-38e366ffd720 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.877234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:58.895131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.895921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.896207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.896486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:58.906390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:58.907857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:58.924837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:58.925692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 259/513] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:58.931619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:58.949216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107fbad-f783-447e-bd5f-acc25230254b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:58.982732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:58.983575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:58.983858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:58.984139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.029135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6684627-dfd3-4e43-85f9-a0940c3b2e1e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.047292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.052049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.053483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.064564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.065351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.065638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.065903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.073112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.074522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.079354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:21:59.089157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:59.090204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:59.091401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 260/514] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:59.091961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.154719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 255/515] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:21:59.155551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.195062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fa3e0-a3b6-4b68-94e5-05f334cd4ff1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.229516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.230204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.230486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.230764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.231512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:59.231822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.233197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.244989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 261/516] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:59.245767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.257032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d2b89-e403-43e6-a029-966c3ba506e5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.281877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.283393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.291575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.292231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.292511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.292785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.293491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.294974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.314389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 256/517] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:21:59 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:21:59.315232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.351580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100140-1208-4a6f-9a01-21470e2c0a50 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.391892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.392685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.392997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.393356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.394091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:59.394398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.395961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.415367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 262/518] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:59.415367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.444448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841883e1-159d-478c-9454-e8f1b0e074d1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.492579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.494043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.494706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.494706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.495914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:59.495914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.497219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.520856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.525316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e7dfb-baf2-4cba-bdc5-6c27e21cc372 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.543109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 257/519] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:59.543776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.567302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.568506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.568980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.569419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.580351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.581998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.585745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:21:59.604668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:59.605617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:59.606471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:21:59.606471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:21:59.606471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:21:59.606471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:59.606471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 263/520] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:59.606749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.655158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d183c-4d02-41fd-8794-1a390d8235bb None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.692499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.693404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.693683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.694084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.701318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.702966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.712327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826327c5-1b28-46b1-8328-1a1b7839e88a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:21:59.718938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:59.719785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:21:59.720632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 258/521] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:59.721096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.751811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.752701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.752989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.753266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.761194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.762727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:21:59.777933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:21:59.779109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 264/522] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:21:59.780181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:21:59.795340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.829322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:21:59.832331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a73a-2e85-479a-a727-128364c6e4ac None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.868091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.868732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.869011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.869286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.901320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae78d1b-d5f6-42d6-9240-53a531dafd21 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:21:59.938311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:21:59.938800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.939025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:21:59.939278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:21:59.939553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:21:59.940275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.940368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:21:59.940581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:21:59.941957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:21:59.953972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 265/523] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:21:59.954762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.031312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.047932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.053384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 259/524] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:00.054169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.066036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd167-f43c-41be-ae26-0d2f65bb9a24 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.103204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.103895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.104173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.104578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.105336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.105638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.107222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.131116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 266/525] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.132403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.157137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6aef7-524f-4b7d-a85d-86b5d9a4ded0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.195078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10291d-789a-498e-91ce-273da148acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.196463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10291d-789a-498e-91ce-273da148acef None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.196899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10291d-789a-498e-91ce-273da148acef None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.197317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10291d-789a-498e-91ce-273da148acef None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.198281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10291d-789a-498e-91ce-273da148acef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.198723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10291d-789a-498e-91ce-273da148acef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.200306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10291d-789a-498e-91ce-273da148acef None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.218966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 260/526] 10.222.0.33 () {56 vars in 1139 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.219521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10291d-789a-498e-91ce-273da148acef None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.238041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a629c3-fca0-4782-a69f-6281a4ec2ae5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.270557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.278143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.278972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.279262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.279519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.283971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.286502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.288217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:00.302649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:00.303732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:00.304768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:00.305806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 267/527] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:00.307289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.336998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10291d-789a-498e-91ce-273da148acef None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.379578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.380281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.380563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.380824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.391989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.393471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:00.411566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:00.412458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:00.413308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:00.413308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:00.413308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:00.413308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:00.413308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 261/528] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:00.413590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.421099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59d51-0e96-4862-92a1-66a1776f2d81 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.460186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.460999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.461344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.461699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.462546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.462885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.464442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.479303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 268/529] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.480283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.538159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e1b13-6c72-4c25-be9d-bab228cdaf2e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.553227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.571971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.591196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.593596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.594198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.594622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.633563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f0121-f1a5-408a-9dcf-2b1599e71a0b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.671541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.672458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.672883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.673314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.674238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.674703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.676299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.687105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.689012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.690505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 269/530] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.691604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.795257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbd5c7-4043-4303-a61f-34de9116f59a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.810698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.815687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 262/531] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:00.816602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.831301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:00.834545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.835550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.836751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.837360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.838603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.839152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.840725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.854109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 270/532] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.855393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:00.937586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9805-24c1-44de-9c64-e1aadafc6cce None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.975179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:00.975378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6221ab-2c04-4882-b9a9-26d2c886a5ff None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:00.976302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:00.977430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:00.978182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:00.979321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=7fb0a7dcacc745229341c1c02c575721) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:00.979831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:00.981484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:00.988374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 263/533] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/role_assignments?user.id=7fb0a7dcacc745229341c1c02c575721&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:00.989427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.041191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.042009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.042290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.042559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.072057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.073555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:01.106770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:01.107697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:01.108547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:01.108547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:01.108547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:01.108547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:01.115197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.117901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 271/534] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:01.123943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.169788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71e38f-345f-4e17-9e92-9c75e54b75a6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.184049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.218726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.219880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.220374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.220841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.230565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.232405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:01.248376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:01.249286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:01.250131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:01.252707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 264/535] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:01.262804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.269572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50e86a-6ed9-4f3c-99f7-eaa6f97677d6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.316304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.317050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.317304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.317578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.318313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:01.318621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.322124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.348767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 272/536] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:01.354468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.443664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4f98-8dff-446b-a9dd-ce1bbfcd4775 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.457312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.474010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee175a30-7829-49e4-9bdf-ea0078328226 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.506569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.507610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.508588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.509004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.509936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:01.510637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.511805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.512445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.512802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.513180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.513552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.514970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.522246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.525375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c38e03f5-4190-4081-b91c-de2ce70a7d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:01.539905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:01.540819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:01.541656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 273/538] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:01.542569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 265/538] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:01.546798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f996-5491-4d36-82b1-74d9bb1d3790 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.546798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f996-5491-4d36-82b1-74d9bb1d3790 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.546798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f996-5491-4d36-82b1-74d9bb1d3790 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.546798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 274/539] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:01.548069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.551687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9f996-5491-4d36-82b1-74d9bb1d3790 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.665525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30e02-cf97-460b-8cc1-6d87920ec66c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.675216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9f996-5491-4d36-82b1-74d9bb1d3790 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.705338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.706380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.706902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.707349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.708450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=6fe6cdefe4174887b82a53df1e2a9fe9) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:01.709028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.711239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cf974-137a-464d-bff5-18110d520d79 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.713840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.714507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.714821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.715454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.715970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:01.716284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.717712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.719185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 266/540] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/role_assignments?user.id=6fe6cdefe4174887b82a53df1e2a9fe9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:01.720633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.721090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.721551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.740008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 275/541] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:01.749219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:01.755856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.767943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:01.769500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:01.889392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda247-0059-4a54-ac58-669ee00f7766 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:01.907638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:01.930411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 267/542] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:01.945341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:01.946536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:01.946864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:01.947138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:01.947977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2dd52fd2b7054851b62ae5b1d4f5ef29) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:01.948287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:01.950083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:01.957009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 276/543] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/role_assignments?user.id=2dd52fd2b7054851b62ae5b1d4f5ef29&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:01.968076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:02.107509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09c2c7-2f70-43c8-a0b9-f135fff25058 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:02.144492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:02.145327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.145607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.145877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.156811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:02.165172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:02.166041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:02.166870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:02.170633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 268/544] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:02.172685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:02.295535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:02.331985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151580c4-1440-4c0c-8947-56718d882b83 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:02.381947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:02.382540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.382959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.383170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.383926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:02.384229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:02.385617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:02.399061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 277/545] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:02.404559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:02.479943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:02.534851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38a904-65d9-4577-9aa3-9b75214d363f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:02.569240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:02.570020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.570318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.570599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.581289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:02.582747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:02.600608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:02.601506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:02.603603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:02.603603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:02.603603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:02.603603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:02.606894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 269/546] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:02.611880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:02.752436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a011449-ac74-41d4-856d-7c48e8dbb0ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:02.784395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:02.790654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:02.791566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.791869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.792145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.792895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:02.793205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:02.794634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae14bf-409a-4434-90a1-d8eaa92f4cac None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:02.814091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 278/547] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:02.819378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:02.865141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce383-f764-4a29-8655-951fce03742e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.865744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce383-f764-4a29-8655-951fce03742e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.866027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce383-f764-4a29-8655-951fce03742e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.868481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 279/548] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:02.873070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:02.873351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:02.873625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:02.899484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:02.900673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:02.994052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:02.998482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207a26-5441-4d5e-83c7-666d0171aaa5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.035764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 280/549] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:22:03.061200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.062019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.062294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.062560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.071514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:03.072950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:03.089419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:03.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:03.098793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:03.099861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:03.104923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 270/550] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:03.108764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.225322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7ac33-6304-4298-a2fe-68a0b032074c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.268976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:03.292111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.292831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.293104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.293369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.311550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa977f-908d-47df-867d-f4339e600d95 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.314608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:03.322420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:03.323354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:03.324332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 281/551] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:03.335820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.354746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.355564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.355845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.356119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.357010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:03.357323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:03.358790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:03.376488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 271/552] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:03.382382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.415200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:03.486850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a315640-173d-4405-9d7a-e21b0ce06a97 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.522574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.523276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.524124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.524398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.608841 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdf45-4a99-4221-90f6-a91ed2418cae None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.618984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:03.620511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003d7c-886f-4251-aa9e-b96b4fdcaf91 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:03.663165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:03.684893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.685568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.685849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.686123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.686903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:03.687213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:03.688601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3eb40-a67a-4c6b-a281-cb88d2f0be36 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:03.706863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 282/553] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:03.708310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94469b-8142-4546-9265-26552c15c897 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.708690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94469b-8142-4546-9265-26552c15c897 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.708971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94469b-8142-4546-9265-26552c15c897 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.715093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 283/554] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:03.715986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94469b-8142-4546-9265-26552c15c897 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.765488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:03.767041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 272/555] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 00:22:03.768350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.768639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.769085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.787948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:03.789221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:03.833358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94469b-8142-4546-9265-26552c15c897 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:03.869225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 273/556] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:03.870145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.890758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:03.891574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:03.891857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:03.892126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:03.905463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:03.906968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:03.924067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:03.924921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 284/557] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:03.926375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:03.935467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:03.993108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6db7a-0cd2-40fc-aeaa-08f523faa923 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.022884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.029917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f4a0b-021b-4fe7-b067-416651dd4211 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.031475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.032867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.033153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.033430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.044768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.046290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.066976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.067842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.068131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.068411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:04.069445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:04.072556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 274/558] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:04.074597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.086799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:04.087858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:04.089938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 285/559] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:04.091977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.186133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.190490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed09b1-5ec2-42df-9637-f4801eafae87 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.198219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a07df-8f0e-48cb-a9fd-df9952574e17 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.234607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.235572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.236067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.236571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.237525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.237976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.238890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.239610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.239895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.240173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.240334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.240955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.241330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.243351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.254244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 275/560] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.255359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.256336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 286/561] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.260105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.278743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.362477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0b43-07af-4c1c-8bcc-ccfc7022979c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.365859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f378-d04c-464f-9d50-6988da119d74 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.401859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.402748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.403060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.403337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.403612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.404096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.404396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.404506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.404677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.404948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.405768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.412091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.413623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.419246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 276/562] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.419246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-bbd11bce-c6b8-4d76-b10f-a5218b98330d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:04.433894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:04.435097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 287/563] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:04.436437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda217-9339-435b-b7a0-ae3b8924af4e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.437137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda217-9339-435b-b7a0-ae3b8924af4e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.437137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda217-9339-435b-b7a0-ae3b8924af4e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.442328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 288/564] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:04.442328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eda217-9339-435b-b7a0-ae3b8924af4e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.450571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.521717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.546317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eda217-9339-435b-b7a0-ae3b8924af4e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.549256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f2f63-5cf4-47c0-b5be-c7184b676a12 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.584141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.584953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.585233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.585506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.588661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.589480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.589763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.590040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.597597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.597797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.599105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.599208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:04.617571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:04.618539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 277/565] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:04.624600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-edea422f-2ced-4244-995c-8a0d64125186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:04.625428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:04.626262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 289/566] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.632632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.645849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:04.647061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:04.711557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.715497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 290/567] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:04.716337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.732227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaf852-7c3e-44ba-9ca4-5495cf3da6f0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.767471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.768132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.768415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.768692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.769535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.769853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.771320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.772100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.788865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 278/568] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.790366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.814654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f9b6-a8d3-4d90-8fb7-be49f4b6e744 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.847860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.848514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.849023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.849150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.849838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.850144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.852186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3650d-fb62-40cb-8cc1-22fd17da5958 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.863772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 291/569] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.864976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282460-6ba6-4281-a821-32425cd24026 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.865259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282460-6ba6-4281-a821-32425cd24026 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.865530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7282460-6ba6-4281-a821-32425cd24026 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.867097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 292/570] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:04.867801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7282460-6ba6-4281-a821-32425cd24026 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.890485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38ab1e-24dd-4b88-89dc-d1d2ea5a3443 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.924069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:04.924787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:04.925068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:04.925347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:04.926055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:04.926365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:04.927755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:04.935663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:04.939782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 279/571] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:04.940703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:04.969958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7282460-6ba6-4281-a821-32425cd24026 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:04.987561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.007355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.008704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.008982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.009249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.019395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:05.027793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.028747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:05.029572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 293/572] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.030357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.050907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba4191-6b88-42b3-9e70-e44e92142838 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.083648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.084424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.084703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.084974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.096071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.097525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-cb31452f-4140-4f7e-91b3-c8ae3fbff380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:05.114872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.115923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.116856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:05.116856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:05.116856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:05.116856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:05.116856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 280/573] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.117635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.117918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.118194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.130910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4b42-26d1-490c-a842-edf78ded5292 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.135359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:05.136548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:05.164390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.165036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.165318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.165598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.166430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=01bd308fead745c7995c1bcf554cec1f) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:05.166744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.168199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.173693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 294/574] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/role_assignments?user.id=01bd308fead745c7995c1bcf554cec1f&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:05.174458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.188292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.212473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 281/575] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:05.213560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.231446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.299476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9faeca-9121-4d85-82fd-db34bb41f18e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.328493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbdb20-27ae-4781-9777-452cde51faf1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.338022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.338863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.339151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.339430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.347971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.349538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:05.365593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 295/576] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.368529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.375604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.376272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.376546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.376811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.377558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:05.377857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.379359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.395089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 282/577] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:05.395089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.440875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.496333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b2b41-d2b7-4ebe-ba75-3db579212cb5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.499530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc40e0-b1f1-44f1-8ab4-6d6281be6e9e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.501457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.532098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.533230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.533708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.534173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.538731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109762d-17bd-461e-973a-76174f58449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.539725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109762d-17bd-461e-973a-76174f58449c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.540059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109762d-17bd-461e-973a-76174f58449c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.540278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109762d-17bd-461e-973a-76174f58449c None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.540997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109762d-17bd-461e-973a-76174f58449c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:05.541468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109762d-17bd-461e-973a-76174f58449c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.543156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109762d-17bd-461e-973a-76174f58449c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.544979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:05.554251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.556150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:05.556965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:05.558263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 296/578] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:05.558263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109762d-17bd-461e-973a-76174f58449c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.565725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 283/579] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.565725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.660669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109762d-17bd-461e-973a-76174f58449c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.667787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da24f-fc6e-45f8-bb13-a33c739e8c80 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.670118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.700466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.701411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.701696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.701970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.702827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:05.703141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.703970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.704559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.704760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.705043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.705318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.716730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.718176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.728098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 297/580] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:05.728098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:05.740448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.741340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.742949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:05.742949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:05.742949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:05.742949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:05.742949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 284/581] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.743240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.757517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.841161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c25bf2-44d1-4e5e-8686-0861ee1d6fa8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.853880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdbf7-78f7-4ee7-9faf-a87d7c8300f4 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:05.877967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.878760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.879103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.879377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.886347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.887804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.889535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:05.890431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:05.890551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:05.890745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:05.891523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:05.891835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:05.893202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:05.905928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 285/582] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:05.906081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:05.907830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:05.908735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:05.909170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.909255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 298/583] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:05.909255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:05.928083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:05.990884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.013435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60b20-637e-4b46-970a-1a2e0193445a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.022916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188fb5-1c9d-42e2-b9d4-dfc434b16c46 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.048248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.048910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.049201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.049509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.050234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:06.050547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.051947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.059857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.060862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.061353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.061827 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.064432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 299/584] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:06.065215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.077809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.079857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:06.097521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:06.098370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:06.099312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:06.099312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:06.099312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:06.099312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:06.100809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 286/585] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:06.102311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.173702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c473-8036-4cc0-b05c-715d58e76979 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.184403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.209116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.209986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.210335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.210662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.219882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.220235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638bb9-df64-4ba9-a7db-f738b5e763b7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.222227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.223874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:06.243038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:06.244139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:06.245107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:06.245107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:06.245107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:06.245107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:06.245107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 300/586] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:06.245515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.263772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.264520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.264902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.265278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.266196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:06.266617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.268337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.288314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 287/587] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:06.288314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.350378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82f83-6f66-4ae3-bda4-99245dc3ca48 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.389548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.390345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.390654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.390952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.395899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdaab6-01b6-4244-bcc2-9bea52cd5760 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.398740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.400221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:06.415228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:06.416168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:06.417082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 301/588] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:06.417672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.430582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.437773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.438586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.439018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.439295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.440065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:06.440368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.441885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.454057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 288/589] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:06.454845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.481727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.533182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9711f3-8d7d-4071-ab5e-38a9d311ac87 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.566136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0e110-24c4-44f2-9299-a0df9c594792 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.568612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.569272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.569548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.569818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.571116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:06.571425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.572789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.584146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 302/590] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:06.589814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.601392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.602284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.602843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.603119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.619442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=42f57540b6cd47559628031b441a1eb9, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.620866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.635882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 289/591] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:06 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:06.636657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.685621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.726571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e45faa-3bb5-4ca5-9b27-9d18298710cf None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.742294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.770715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.771417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.771697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.771972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.772835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:06.773144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.775567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.780059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84993-d340-4d51-a70a-fe69359d6ef2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.796641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 303/592] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:06.797161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.831504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.832593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.833108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.833389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.842413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.844373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:06.862495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:06.863467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:06.864348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:06.871005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 290/593] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:06.872058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:06.908999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1ffd-a44d-4b21-93c8-9e07e8fcbbc2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.946924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:06.947600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:06.947954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:06.948300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:06.956973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:06.958530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:06.976054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:06.976678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:06.977886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:22:06.978856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:06.980635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1750b2-f7ab-47bd-beb4-1c76de272dae None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:06.982252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 304/594] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:06.983262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.028915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.030924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.031393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.031847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.045824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.047974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.055313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:07.072464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:07.073385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:07.074277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:07.074277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:07.074277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:07.074277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:07.076611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 291/595] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:07.077600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.119359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0056dbd-0832-4009-a12c-f32dcce5895c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.159700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.160365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.160644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.160918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.161792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:07.162103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.163650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.176020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 305/596] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:07.182196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.217292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd46fc-3037-4202-bae0-f86bdb3768b7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.262524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.263273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.263656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.263940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.264808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:07.265125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.266620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.292490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 292/597] 10.222.0.33 () {56 vars in 1139 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:07.293139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.293481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.308087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.309883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd206615-2260-4a12-a2d2-cd7a9edbfc57 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.353518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.354183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.354466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.354737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.355491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:07.355799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.357174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.373985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 306/598] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:07.373985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.412682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405249e-3aa3-4824-8b43-fd15094df79c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.449914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.451173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.451746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.452018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/7fb0a7dcacc745229341c1c02c575721/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.468113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=7fb0a7dcacc745229341c1c02c575721, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.469671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74269ac0-f4c7-406a-9074-982b31100db9 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.477400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98e7d-13b7-4d90-8cd7-9b85a5e197b6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.487234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 293/599] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:07 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/7fb0a7dcacc745229341c1c02c575721/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:07.489126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2b196-27c9-4c22-a16f-2cd0a4258537 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.489417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2b196-27c9-4c22-a16f-2cd0a4258537 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.489699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2b196-27c9-4c22-a16f-2cd0a4258537 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.491419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 294/600] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:07.495838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2b196-27c9-4c22-a16f-2cd0a4258537 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.532838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.534025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.534549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.535084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.554097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.556081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-461a2393-38f3-4c09-9beb-dd2d70ddcfd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:07.576612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:07.577613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:07.578546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:07.578546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:07.578546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:07.578546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:07.585490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 307/601] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:07.587959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.588461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.588935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.592756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.596094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.613822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:07.615474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:07.655392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2b196-27c9-4c22-a16f-2cd0a4258537 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.697309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 308/602] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:07.697309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.701414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.702260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.702545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.702869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.712307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.713527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:07.729977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:07.730953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:07.732290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 295/603] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:07.735707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.856213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c438-4daf-4f3f-acfe-59a22cf93c82 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.860073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef286-c8e9-4ed1-ab86-7499560638f6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:07.888675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.889997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:07.895952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.896889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.897281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.897648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.898500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:07.899019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:07.900601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:07.908863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:07.910210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:07.910495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:07.910786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:07.913344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 309/604] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:07.914328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:07.923760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:07.933801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:07.935116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:07.936979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 296/605] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:07.938681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.043051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89770bb-fd03-413a-adcb-fa84b93d5ebe None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.043350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48fc2b-ed24-4445-ac16-a85f540ed283 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.078082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.079053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.079429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.079794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.086263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.087072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.087274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.087395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.087621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.088414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:08.088725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.089168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.090355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:08.104468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:08.106265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:08.108139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 297/606] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:08.109174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.110193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 310/607] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:08.110193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.151055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.174109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.223218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844900e8-7aef-4358-85c0-90759cb47a11 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.233828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c98fc-823f-49a2-b8ff-d45172cf0ef0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.271333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.271572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.272224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.272342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.272555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.272726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.272862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/6fe6cdefe4174887b82a53df1e2a9fe9/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.273155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.273968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:08.274331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.275793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.285201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 298/608] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:08.286114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.291272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=6fe6cdefe4174887b82a53df1e2a9fe9, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.292848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.312112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 311/609] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:08 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/6fe6cdefe4174887b82a53df1e2a9fe9/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:08.313123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.446896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f7f99-effe-42fd-a822-53c51fa982fc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.448654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.493965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.493965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.493965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.495014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.507064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.508664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.511550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab55bf-9f06-4a19-b389-1a3054223126 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.513485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:08.531514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:08.532490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:08.533386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:08.533386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:08.533386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:08.533386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:08.533793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 312/610] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:08.540756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.576348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.577079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.577360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.577633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.579028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:08.579347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.581022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.596360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 299/611] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:08.602006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.711800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50105cc-5bb1-44f2-ab09-c99ac896f465 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.744594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba142a1c-ea43-463b-944e-08582f48a7fb None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:08.764799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.766137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.766799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.768112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.770252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:08.771425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.774150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.787570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 313/612] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:08.793006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:08.797885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:08.800746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.807666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.808289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2dd52fd2b7054851b62ae5b1d4f5ef29/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.828529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=2dd52fd2b7054851b62ae5b1d4f5ef29, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:08.830568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9ced3-9074-4dc8-a676-3e92f7726743 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:08.841425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.858253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 300/613] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:08 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2dd52fd2b7054851b62ae5b1d4f5ef29/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:22:08.902337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0373-58c3-4f5d-b72d-f24e2529083e None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.902637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0373-58c3-4f5d-b72d-f24e2529083e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.903105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0373-58c3-4f5d-b72d-f24e2529083e None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.904860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 301/614] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:08.910521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005405-08bb-4378-8329-579928dd58dc None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:08.910914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005405-08bb-4378-8329-579928dd58dc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:08.911267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005405-08bb-4378-8329-579928dd58dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:08.929164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-55005405-08bb-4378-8329-579928dd58dc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:08.930151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-55005405-08bb-4378-8329-579928dd58dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:08.941306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:08.970739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601df6b-4150-4ce9-9ca6-1bb74dad7e27 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.032200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.033066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.033345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.033611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.050756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.055013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.077114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 302/615] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:09.080244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:09.081181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:09.082135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 314/616] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:09.089300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55005405-08bb-4378-8329-579928dd58dc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.103032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.200754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:09.221718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55005405-08bb-4378-8329-579928dd58dc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.236837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751fc9b-2d9a-4d82-9115-b3869f640db9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.261731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.262943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.263452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.263931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.264965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:09.265471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.267110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.282494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:09.282697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.284217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.284805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.286300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.287700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 303/617] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:09.293345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.298947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:09.307909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:09.308877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:09.309710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 315/618] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:09.316471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.446588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60171912-0b3e-4719-addc-28c3195a58c7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.486076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.488676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.488676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.488676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.488676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:09.489129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.492874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.492874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:09.503257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 316/619] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:09.510836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.567898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e898-1dab-4e21-90a6-31fc1adf56db None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.627471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f8ff-7084-4fbe-a31a-7b5e34f408c8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.633861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.635210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.635637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.636056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.664392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.666138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.666589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.670575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.670575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.670575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.680484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.682061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:09.701742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:09.702881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:09.703977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:09.703977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:09.703977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:09.703977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:09.705521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 317/620] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:09.714831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:09.716039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:09.717100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:09.718204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:09.718204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:09.718204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:09.718204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:09.719862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 304/621] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:09.725373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.774179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:09.834484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:09.843608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc33a-62d9-46c4-bec0-003a9131f6ab None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.888211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.889266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.889758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.890275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.891617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:09.891930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.893307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6339-9085-40ce-b5e0-4fc64dd6e204 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.907438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24116c48-af95-4af9-97e3-e2b985c4e6de None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:09.909178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 318/622] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:09.915131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b10c-840b-48ee-982f-8fb79c0a0cc9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.915131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b10c-840b-48ee-982f-8fb79c0a0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.915131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b10c-840b-48ee-982f-8fb79c0a0cc9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.915131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 319/623] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:09.916780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052b10c-840b-48ee-982f-8fb79c0a0cc9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:09.952162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:09.953300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.953809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.954302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.955362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:09.955951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:09.957756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05c414-686a-45f0-80f3-2eebaec47353 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:09.971926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 305/624] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:09.973589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:09.974074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:09.974554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:09.993631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:09.995255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:10.037399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052b10c-840b-48ee-982f-8fb79c0a0cc9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.047622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.069196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 306/625] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:10.070665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.078708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.080304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.080888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.081818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.096091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.098883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:10.120675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:10.122476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 320/626] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:10.123459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.137366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.208565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84676a-fb6b-4d76-88d8-eecacf1fb6f8 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.250882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.251962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.252175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.252546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.259693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2f5ad-d578-479a-a4d4-7669adad235d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.260232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.261793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:10.276744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:10.277867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:10.278985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:10.280681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 307/627] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:10.285931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.308056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.309095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.309520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.310459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.326360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:10.337590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:10.338846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:10.339911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:10.349389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 321/628] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:10.349389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.384434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.417081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc2e2-e4c1-481b-88e3-4ebd30f9d502 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.418946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.458802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.459412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.459696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.459966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.461408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:10.461716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.463176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.492427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 308/629] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:10.493315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb9b4d-ac70-4989-bdb4-0cbea1242446 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.494348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.540314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.542395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.542928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.543425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.545089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:10.545751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.547782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.567179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 322/630] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:10.567968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.613698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d444-fd27-45a3-8097-5f177e2150d1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.658621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.659728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.660309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.661276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.661770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.663039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:10.663789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.665865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.674214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab9736-b8f2-4ba8-8d6a-6339bea5ce24 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.682355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 309/631] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:10.683744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.711124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.727087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.729010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.729606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.730163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.801341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf47ae-76d4-44d8-9c31-36c5e22b1d90 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:10.813460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.815435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a783ae-067c-4b48-9df3-d273f49125c9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.836573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:10.837795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.838284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.838749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/01bd308fead745c7995c1bcf554cec1f/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.854739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=01bd308fead745c7995c1bcf554cec1f, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:10.856416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:10.870983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 310/632] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:10 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/01bd308fead745c7995c1bcf554cec1f/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:10.872420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.915418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:10.934496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 323/633] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:10.940574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb8eb-7340-40a1-8278-95ea2b725389 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:10.940861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb8eb-7340-40a1-8278-95ea2b725389 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:10.942519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cb8eb-7340-40a1-8278-95ea2b725389 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:10.948079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 324/634] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:10.949040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cb8eb-7340-40a1-8278-95ea2b725389 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:10.977147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.024400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8fe05-e258-492f-983d-9a5ebf638584 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.076370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cb8eb-7340-40a1-8278-95ea2b725389 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.091047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.091742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.092023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.092359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.115020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.115815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.116104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.116361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.116953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.118353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.127224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.128880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b1fa6a0e-971d-4d0c-887f-6f255abfd211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:11.151419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:11.152361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:11.153247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:11.153247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:11.153247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:11.153247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:11.153247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 325/635] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:11.153995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:11.154851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:11.156378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:11.156378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:11.156378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:11.156378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:11.156611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.156611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.156611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.158970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 311/636] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:11.160296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.174212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:11.175792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:11.222333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.260758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 326/637] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:11.261975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.296760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d2d4b-92f3-492d-8e71-9012ad9b2afc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.330394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.343935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.344537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.344818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.345094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.345842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:11.346154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.348005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.365521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 312/638] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:11.366260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.380705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acff70-358b-4c35-9c3f-eaa214c35f7f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.419211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.420822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.421085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.421356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.422207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:11.422522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.423944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.437469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 327/639] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:11.438264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.470107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.482584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761dff8-60a1-4d93-8b50-14451ef70287 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.524597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.525610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.525724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.525921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.536404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:11.543656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8dc76-3d62-48b0-8145-cb1e286f1d78 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-aed73194-ef75-460c-9f39-63e6ca817e98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:11.544201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:11.545103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:11.545981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 313/640] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:11.547393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afacdbb-f8bb-4d5f-a0ce-8e6687327805 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.547393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afacdbb-f8bb-4d5f-a0ce-8e6687327805 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.547393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afacdbb-f8bb-4d5f-a0ce-8e6687327805 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.548707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 314/641] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:11.549263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afacdbb-f8bb-4d5f-a0ce-8e6687327805 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.588268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.589059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.589333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.589599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.596367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.597760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.600091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:11.613275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:11.614261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:11.615215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:11.619664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 328/642] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:11.620324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.663348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afacdbb-f8bb-4d5f-a0ce-8e6687327805 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.699669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.700462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.700745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.701021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.708679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.710139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.719644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-601dd233-ea00-492d-920e-55db6c3c3379 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:11.725772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:11.727290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:11.728558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 315/643] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:11.729260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.730021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.730021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.736149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19580049-3262-44cd-9d08-2aa9b8aaa319 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.749592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:11.750816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:11.776579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.777559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.778136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.778385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.779332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:11.779641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.781557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.796350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 329/644] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:11.796350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.826803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 316/645] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:11.826803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.859205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:11.917369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea62ec-3276-4383-aaeb-0171bf5a1373 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.930041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428912c4-f978-4ded-a4dd-8f131a8d9ad3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:11.952840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.953688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.954040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.954316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.955130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:11.955556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.957080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.964472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:11.965141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:11.965421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:11.965698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:11.966539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:11.966878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:11.968292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:11.971524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 330/646] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:11.972510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.987388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 317/647] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:11.988205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:11.994896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.088085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.090217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d44ce-e942-481d-859f-989d7797a7ac None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.100565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822fc0f-4d0a-443a-b117-7c1dbe8ba246 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.130452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.131574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.132157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.133155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.136224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.137013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.137292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.137560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.145251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.146846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.147819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:12.155650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:12.156752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 318/648] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:12.157592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:12.172229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:12.173312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 331/649] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:12.176024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.245476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.273341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f85e3-167c-4911-ad28-dc12b5dcb6a7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.285167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c638eb-4d09-4b04-bae2-5cb070bd85ed None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.308277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.308921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.309206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.309486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.310314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:12.310629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.312165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.318519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 319/650] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:12.319263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.331601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.332670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.333153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.333636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.345259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.424773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.426697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.461343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133d52-fb62-42b0-b903-cec5c20ae2af None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.507936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.520941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.520941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.520941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.520941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.521356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:12.521356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.524390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.533546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 320/651] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:12.534530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.557288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 332/652] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:12.558729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.646756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.671168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafb4a-cc88-42e7-8e69-1f5cdd43725d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.675349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66f263-7214-4eba-8e96-780d4ec3d965 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.711765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.712993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.713738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.714122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.718721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.719577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.720005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.720424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.721306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:12.721757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.723368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.725697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.728367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.738946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 321/653] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:12.743771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:12.751742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:12.752653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:12.753647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:12.753647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:12.753647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.753647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:12.753647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 333/654] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:12.759050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.780550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.863227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25bf6-cdd2-412f-9cee-642f3d49e569 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.895408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98dd8-f62d-4cf0-9287-79a6b0099661 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:12.901904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.903057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.903542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.904014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.912196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:12.916483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.918884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.938143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:12.938885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:12.939163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:12.939432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:12.940754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:12.941613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:12.942536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:12.942536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:12.942536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:12.942536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:12.944582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 322/655] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:12.946112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:12.953054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:12.954520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:12.973868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:12.974897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 00:22:12.975847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:12.978931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 334/656] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:12.979954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.069930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d50e28-d389-4080-b492-4a4157dddbe7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.089288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.110919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.111639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.111928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.112206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.112926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:13.113233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.114663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.117499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a4f51-f816-4879-bffb-4fe011800e28 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.127310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 323/657] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:13.128122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.158166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.158861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.159146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.159421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.160149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:13.160460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.161970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1672-3f80-49e5-ba0e-91a560991f6d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.174128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 335/658] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:13.175406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea0df-c869-4145-9830-bc2e2face1a8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.175687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea0df-c869-4145-9830-bc2e2face1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.175961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea0df-c869-4145-9830-bc2e2face1a8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.177336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 336/659] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:13.179949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ea0df-c869-4145-9830-bc2e2face1a8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.186299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.245939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0677ae-001c-475b-adc4-31930af8a86d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.281330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.281994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.282279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.282553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.283416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:13.283731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.285155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.288295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ea0df-c869-4145-9830-bc2e2face1a8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.306881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 324/660] 10.222.0.33 () {56 vars in 1137 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:13.306881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.330095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.330811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.331089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.331357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.334138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.341433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.343257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b844881d-f9fa-49cd-b4c6-1b3cdb90eed0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:13.359115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:13.360079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 00:22:13.361024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:13.364871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 337/661] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:13.366108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.366392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.366756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.383915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:13.385165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:13.412050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d00360-a5ff-4f62-82c3-595f3cf4a9a2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.423061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.448516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.449305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.449578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.449932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.456778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 338/662] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:13.457436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.459060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.461131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:13.479978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:13.480920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:13.481896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:13.485790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 325/663] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:13.486607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.572038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40745-31f1-4497-aae5-1065aabcf7ad None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.594858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ccb99-f9ac-4e06-84a0-39d3aa9baa9a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.598240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.609338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.609985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.610263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.610535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.630744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.631437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.631715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.631985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.633447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:13.633752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.635200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.656779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 326/664] 10.222.0.33 () {56 vars in 1139 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:13.656779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.683850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.697214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.698879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.760509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3e281-a6c5-417b-819e-a157a6ff16d1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.795243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.796021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.796298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.796570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.801998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 339/665] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:13.801998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.813184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:13.822282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:13.823454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:13.824474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:13.825297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 327/666] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:13.826058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.831895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.904569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcabe0-3abb-4a0a-8cd8-28e43c401f63 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.915813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:13.933266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fec8a-d209-4863-8799-e7a205a8c682 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:13.938142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.938901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.939245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.939585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:13.940438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:13.940829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:13.942313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:13.959213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 340/667] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:13.959213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:13.975536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:13.976800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:13.977177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:13.977683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.047626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.053375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.055500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.070932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab063c25-a597-4fa4-8a7f-19c3af7b1588 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.108385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.109424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.110295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.110746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.122357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.124042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:14.141540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:14.142654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:14.143693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:14.143693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:14.143693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.143693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:14.143693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 341/668] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:14.145521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.148349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.163405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 328/669] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:14.164873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.266964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1836c1-dd3d-406d-999e-9927fe4fd86d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.274559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff3cfa-8360-4f4f-b2df-1a718da7b80a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.302366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.303068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.303348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.303623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.304933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:14.305239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.306616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.311277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.312040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.312325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.312599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.312859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.313690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=7fb0a7dcacc745229341c1c02c575721) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:14.313814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.315403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.316710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 342/670] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:14.317582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.325040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 329/671] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/role_assignments?user.id=7fb0a7dcacc745229341c1c02c575721&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:14.326097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.400075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.445269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb097-8ba4-4913-a40b-81190d08a6c0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.477215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469bc8f-7ca2-45d9-8a5c-a301ee590125 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.486149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.486866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.487154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.487430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.488172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:14.488479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.489977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.508601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 343/672] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:14.509601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.530563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.531410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.531693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.531967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.545463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.546961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:14.566462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:14.567462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:14.568669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:14.568669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:14.568669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.568669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:14.568669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 330/673] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:14.569023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.596077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.639497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b43f4-dcef-4aa6-9d49-3e0484176d2a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.675615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094827a1-d875-4451-b764-cfb80bcf8d78 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.677059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.678518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.679487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.680043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.680926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.692112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.693777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:14.712795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:14.713763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:14.714643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:14.714643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:14.714643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:14.714643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:14.715603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.715603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.715603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.715603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.718624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 344/674] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:14.719499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.723608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.725095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-1bd50201-83cd-4e00-ace9-8b60ca39c967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:14.740615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:14.741704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:14.742642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 331/675] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:14.743382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7359-6113-41fd-8522-33b6500f9b9a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.743673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7359-6113-41fd-8522-33b6500f9b9a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.743946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7359-6113-41fd-8522-33b6500f9b9a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.748993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 332/676] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:14.748993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b7359-6113-41fd-8522-33b6500f9b9a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.838883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb967a-8aef-4835-94f3-4c2c3b928028 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.853365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:14.858254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b7359-6113-41fd-8522-33b6500f9b9a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:14.878040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.878832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.879092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.879359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.880076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:14.880377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.882818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.896304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:14.896969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.897249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.897532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.898821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:14.898943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 345/677] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:14.899136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:14.899695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:14.900510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545de43-085a-474e-a13a-e1f69de50820 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:14.913247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 333/678] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:14.914517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:14.914819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:14.915092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:14.931421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:14.932506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:14.950380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.004846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 334/679] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:15.005913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.033265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd245b-8924-4177-b6c1-92ce07768e80 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.070915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.071915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.072400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.072814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.073865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.074402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.076021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.090923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 346/680] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.091809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.119003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d568-3ee3-4e51-bcc9-6c45687f170e None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.128345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.155937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.156734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.157080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.157409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.164670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.166204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:15.182680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:15.183862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:15.184805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 335/681] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:15.185291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.202444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.206536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299643bb-f12f-47b2-8998-17b70aad7fb0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.243573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.244497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.244877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.245245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.252348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.253897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:15.268668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:15.269780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:15.270989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:15.272297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 347/682] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:15.273614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.294002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d698c64-1198-4e69-8c65-b21b89005db8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.328278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.329061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.329337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.329607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.339932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:15.347478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:15.348415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:15.349264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 336/683] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:15.349975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.373977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.384871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d75dd-42be-4361-97ff-8ac57ff4aa10 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.420004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.421130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.421410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.421824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.422663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=6fe6cdefe4174887b82a53df1e2a9fe9) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.423010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.424481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.424950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.438881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 348/684] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/role_assignments?user.id=6fe6cdefe4174887b82a53df1e2a9fe9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.438881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.472805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb788ca-d50b-40df-aefc-dcd7a89a4fcd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.527591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.528694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.529072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.529691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.531353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.531776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.533361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.549638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 337/685] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.550512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.587953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1c9b-be47-49c2-a897-3c0ccfb87012 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.637231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.638614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.639086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.639549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.640567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.640876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.642281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.662596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 349/686] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.664285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.673618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e8072-27cf-4fb3-93fa-aec17604b294 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.684668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.716912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.723867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.724270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.724834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.725483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.728976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.729365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.730848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.740519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 338/687] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.745540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.802771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a3ba8-1546-4656-b645-7ca5f6815a9b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.848411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.849067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.849344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.849617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.850453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2dd52fd2b7054851b62ae5b1d4f5ef29) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:15.850763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.852119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97182b-dbfe-41a2-8277-8d743212f077 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:15.853197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:15.865951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 350/688] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/role_assignments?user.id=2dd52fd2b7054851b62ae5b1d4f5ef29&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:15.865951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:15.894175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:15.894868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:15.895333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:15.895604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:15.968089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:15.976912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:15.978920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.074324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.098815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 339/689] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:16.107952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.111523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731172e-b946-4717-b9cf-2e7a6c128bfd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.151566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.152734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.153210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.153693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.167580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.169380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:16.189816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:16.191035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:16.192128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:16.192128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:16.192128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:16.192128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:16.193634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 351/690] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:16.198981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.224050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.238889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff48d6b-4176-4127-8a23-3ccc1271dcca None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.281727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.282370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.282748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.283174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.293923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.295642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:16.314470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:16.315639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:16.316752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:16.316752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:16.316752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:16.316752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:16.318006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 340/691] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:16.323867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.350008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab4902-6bd3-4b18-ad90-53bf056eb5f8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.381674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.388563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.389531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.390009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.390456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.391657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:16.392429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.393973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.407594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 352/692] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:16.413572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.572428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789d91-2932-4f4d-9ff9-ef41ece82135 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.583876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd789ef-0a61-47e6-98d2-b12e8fc697a5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.621553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.622623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.623074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.623577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.626426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.627218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.627500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.627773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.628498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:16.628802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.630170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144949f-d9df-46cd-b95d-e910c0d49e27 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.633796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.635498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.648533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 341/693] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:16.649948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079de7c-bdc3-441f-b1a8-7cdee7e95f5d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.650245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079de7c-bdc3-441f-b1a8-7cdee7e95f5d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.650513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079de7c-bdc3-441f-b1a8-7cdee7e95f5d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4ac680ec-7329-4a49-a9f8-5cb9456d32ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:16.651853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:16.653213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:16.654294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:16.654871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 342/694] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:16.654871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079de7c-bdc3-441f-b1a8-7cdee7e95f5d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.656687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 353/695] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:16.658086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.658581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.659088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.662369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.679452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:16.680812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:16.733820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.762154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 354/696] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:16.763709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.773333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079de7c-bdc3-441f-b1a8-7cdee7e95f5d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.810750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.811567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.811941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.812309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.821586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.823424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:16.839369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:16.840477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:16.841626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:16.848082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 343/697] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:16.849022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:16.898132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0853cc-47b7-462f-a09f-9801e0b9dbc8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.924306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:16.947451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:16.948184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:16.948390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:16.948660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:16.949496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:16.949803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:16.951257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:16.973574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 355/698] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:16.974327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fca86-a7d8-429e-af8b-3f0d92f43fab None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:16.974441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.011907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.013027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.013027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.013341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.013894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.015403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:17.029615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:17.030585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:17.031494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 344/699] 10.222.0.33 () {56 vars in 1129 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:17.032359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.033829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.117647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde7c3-5fb5-4fd2-9997-8d65df3c6af2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.137737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb076958-4102-432c-85db-efe0aa7d7efd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.159504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.160297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.160679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.161050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.161933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:17.162339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.164448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.173659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.177272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 356/700] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:17.178312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.181516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.182511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.182956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.184173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.265536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.267272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.281127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.286185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf56af-0b2c-4130-8329-cc4a8079b404 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.324723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.325569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.325925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.326300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.327169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:17.327568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.329034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.334435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 357/701] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:17.335970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.378286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 345/702] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:17.379554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.410690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.447873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830bb11-b376-445b-9d25-1af2c9b3fd4a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.485160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.485818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.486095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.486367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.487249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=01bd308fead745c7995c1bcf554cec1f) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:17.487558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.488693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06fd2c-6158-49b4-8796-b182d4bbaa9d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.489038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.495673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 358/703] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/role_assignments?user.id=01bd308fead745c7995c1bcf554cec1f&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:17.496488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.526941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.528888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.529841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.530366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.530829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.531802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:17.532317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.533980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.541440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 346/704] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:17.619728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5c92-7cc5-4150-a806-574b6c1f6028 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.659533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.660339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.660616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.660883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.673771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.675336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:17.694703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:17.695677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:17.696575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:17.696575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:17.696575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:17.696575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:17.696575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 359/705] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:17.701734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.821795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:17.828397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecee057-0dcc-400d-9bf8-5b3e66ff549c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:17.870704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:17.871397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:17.871674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:17.871944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:17.872667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:17.872966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:17.874326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688a361-fd07-497e-8334-6fccb41666d7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:17.887997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 347/706] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:17.893036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:17.990312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:18.030550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54a9c-1157-4ae9-9d55-46400eddacbc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.064844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d2ef0-baf3-472d-a968-fde279b6d26b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.065491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d2ef0-baf3-472d-a968-fde279b6d26b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.065971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d2ef0-baf3-472d-a968-fde279b6d26b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.068906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 348/707] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:18.073977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.074194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.074470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.082228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.083358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.083843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.084324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.093685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.095490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:18.109120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:18.110315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:18.111947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:18.112808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:18.113700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:18.117921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 360/708] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:18.123074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.242877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 349/709] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:18.267133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d714-3768-4da4-aad4-675afba5a431 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.315403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.316415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.316919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.317408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.318403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:18.318933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.321214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:18.328650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:18.331264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.335648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 361/710] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:18.340984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.477796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:18.499877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322970c-2dc5-4f41-854b-e215bb5bc360 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.510800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be6499-d3fa-45e4-b269-6cdaf5af1b79 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.539864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.540862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.541221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.541564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.549274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.549940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.550223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.550503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.551378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:18.552091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.553570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.553788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b92df-1800-4e37-9c91-98cd17a94b4a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:18.555811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:18.561012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 362/711] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:18 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:18.562369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991ac1b-77f6-443b-8c26-869d32eaf1ac None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.562661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991ac1b-77f6-443b-8c26-869d32eaf1ac None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.562962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991ac1b-77f6-443b-8c26-869d32eaf1ac None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.565167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 363/712] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:18.572409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.572775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.573129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:18.585663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:18.586668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:18.587715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:18.587715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:18.587715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:18.587715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:18.587715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 350/713] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:18.594188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.619882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:18.621077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:18.720357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858582-4fac-4335-bcf9-c62ee56a3431 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.762600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.763303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.763740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.764031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.764770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:18.765078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.766463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:18.769753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:18.786920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 351/714] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:18.788804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.791566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:18.815287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 364/715] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:18.851584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:18.905601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7d7c-5732-4978-931f-25bbcd752532 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:18.953763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:18.954442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:18.954725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:18.955068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:18.955804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:18.957269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:19.017698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 352/716] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:22:18 2022] POST /identity/v3/users => generated 254 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:19.045015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa7fbd-f509-47d8-8199-ed001be47653 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:19.098617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:19.099471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:19.099754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:19.100029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] PATH_INFO: `/v3/services/network` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:19.100767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:19.103021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:19.121398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:19.122475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:19.123430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:19.127602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 365/717] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:18 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:19.128944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:19.215605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:19.271499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fddb7-47af-423a-a2eb-a4586b9c2d80 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:19.307707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:19.308364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:19.308638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:19.308909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:19.309606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] RBAC: Adding query filter params (name=network) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:19.309909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:19.311427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44caf-4b6f-4b7d-b46b-ddc3fbca5c2b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:19.316422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 353/718] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:19.322021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:19.383478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:19.430015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c6c-133f-4a02-97ba-52a7ad9e930a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:19.465073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:19.465734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:19.466010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:19.466280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:19.467030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:19.467335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:19.468705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0ee07-c3f1-434c-bca6-675f26363ab3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:19.474864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 366/719] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:20.161151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27afa-4c49-44ce-8c58-b68739483339 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.161680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27afa-4c49-44ce-8c58-b68739483339 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.161680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27afa-4c49-44ce-8c58-b68739483339 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.163848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 354/720] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:20.169142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681430-6d98-4a0a-9362-e1e9239fe02a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.169538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681430-6d98-4a0a-9362-e1e9239fe02a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.169721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681430-6d98-4a0a-9362-e1e9239fe02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.191995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6f681430-6d98-4a0a-9362-e1e9239fe02a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:20.193174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f681430-6d98-4a0a-9362-e1e9239fe02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:20.291891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 367/721] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:20.317039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99923-aa0f-4041-acd5-4ef259562edb None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.317039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99923-aa0f-4041-acd5-4ef259562edb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.317039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99923-aa0f-4041-acd5-4ef259562edb None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.317711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 355/722] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:20.323719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.324009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.324288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.332992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99923-aa0f-4041-acd5-4ef259562edb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:20.349664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:20.350918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:20.460393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99923-aa0f-4041-acd5-4ef259562edb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:20.484827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:20.509903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:20.510933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.511447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.511929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.512871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:20.515009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:20.520744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:20.533858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 356/723] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:20 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:20.582235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 368/724] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:20.620792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:20.745234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a78-a28b-467c-b39c-1c2d986b025c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:20.779938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:20.780729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:20.781009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:20.781284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:20.792486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:20.800352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:20.801279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:20.802119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:20.806943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 357/725] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:20.811883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:20.964933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9429bd-b66b-4cc3-a3d2-1c0d95cebb36 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.016115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.016711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.016992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.017259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.018004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:21.018305 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:21.019708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43e909-f79d-4641-8c6c-718d7995b88e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:21.034852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 369/726] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:21.041643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.112047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc64b-07be-40eb-81e9-50f724b13c7e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.112394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc64b-07be-40eb-81e9-50f724b13c7e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.112595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc64b-07be-40eb-81e9-50f724b13c7e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.114193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 370/727] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:21.121155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.121434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.121703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.130819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:21.152228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:21.153426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:21.219140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:21.239067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb9ef4-fab7-44d1-ab06-21e76733cd50 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.276729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.277912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.278427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.278937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.290669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:21.292596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 371/728] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 00:22:21.292907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:21.322802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:21.330353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:21.331251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:21.332154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:21.332154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:21.332154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:21.332154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:21.339275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 358/729] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:21.340830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.418329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c1425-3efd-4227-bacc-1ffffad6e5a9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.463540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.465694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.466355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.467993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.489477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:21.502697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:21.503663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:21.504493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 372/730] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:21.510356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:21.513302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.532311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a4ee-01d1-4358-b81c-41e89c2cdacc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.568837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:21.596412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.597088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.597354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.597690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.598345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:21.598648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:21.601039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:21.629247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 359/731] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:21.632813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.703926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45600620-770c-47ca-9f4d-997b385c7790 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.768016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.768690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.768971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.769245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.770002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:21.770312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:21.771715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:21.787899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 373/732] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:21.793988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.821205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322fcd6-9434-4944-b6e1-873b6287604a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:21.887689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:21.888483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:21.888763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:21.889043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:21.897652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:21.899477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:21.915938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:21.916918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:21.917849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:21.924976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 360/733] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:21.932409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:21.951128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:21.987141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.003043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a366d-54f1-4384-9324-78eeb22b9ff5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.055429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.056465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.056747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.057019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.074173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.075993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.094548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d2f-c569-4d53-af1e-249ef9ccc23f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1415a8fc-eb09-463f-86c4-3a68d8a971ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:22.099044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:22.099910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:22.100830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:22.100830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:22.100830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:22.100830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:22.100830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 374/734] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:22.101893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703797-0180-44a0-af30-e5481aabbcfd None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.102881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703797-0180-44a0-af30-e5481aabbcfd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.103171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703797-0180-44a0-af30-e5481aabbcfd None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.106876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 375/735] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:22.109821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60703797-0180-44a0-af30-e5481aabbcfd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.138418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.139383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.139724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.139932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.140763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:22.141149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.142650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ecbad-e185-4544-bc2d-17a5fbf23a81 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.160784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 361/736] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:22.160784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575728-9bcb-4188-8b86-c94a67d92443 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.160784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575728-9bcb-4188-8b86-c94a67d92443 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.160784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec575728-9bcb-4188-8b86-c94a67d92443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.181066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ec575728-9bcb-4188-8b86-c94a67d92443 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:22.182239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ec575728-9bcb-4188-8b86-c94a67d92443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:22.224160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60703797-0180-44a0-af30-e5481aabbcfd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.246461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.255481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 362/737] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:22.257097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a47e0-7299-4efc-952b-f0868e70e206 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.257531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a47e0-7299-4efc-952b-f0868e70e206 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.257974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a47e0-7299-4efc-952b-f0868e70e206 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.259686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 363/738] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:22.260618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a47e0-7299-4efc-952b-f0868e70e206 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.276672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.277505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.278044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.278319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.279398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:22.279943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.281680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93462ba-b910-4982-bd3c-7d50c0648d54 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.296453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 376/739] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:22.297959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.298249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.298525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.304913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.318119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:22.319426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:22.375225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a47e0-7299-4efc-952b-f0868e70e206 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.388093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 377/740] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 00:22:22.388905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.423338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.424408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.424993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.425548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.501208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.508381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e36c6-af1a-4871-8560-ede9a8d7c75a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.512379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.513851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.541379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.547738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.548569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.549016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.549443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.550191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.551609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.569880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 378/741] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:22 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:22.570692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.633526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 364/742] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 00:22:22.634839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.695338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407e90c-d481-474f-af80-6cc091fcbe5e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.732769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.733623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.733836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.734107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.745012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.746461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.748773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.755269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816acb-e104-4e98-b3c4-82be89c0ecaa None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:22.762427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:22.763400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:22.764340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:22.768236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 379/743] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:22.769071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.806571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.807795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.808213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.808628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.820820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:22.829272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:22.830198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:22.831097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:22.833814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 365/744] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:22.839194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:22.839902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:22.890907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e585-0060-40e8-ab5c-abc3189d69d8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.935950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:22.936896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:22.937249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:22.937590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:22.955702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:22.957285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:22.972793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a35ae-260c-43b6-8522-46e3b681dd95 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:22.977074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:22.978207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:22.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:22.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:22.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:22.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:22.983309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 380/745] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:22.984288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.008605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.009270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.009574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.009877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.010885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.011202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.012751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.031438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 366/746] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.032146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.048811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.093519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.096568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943d8c4-f0fb-4476-976a-596b8183be13 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.132970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.133632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.133912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.134183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.134954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.135262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.136631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faedd7e-340b-4861-a90b-8714db6caf69 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.145549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 381/747] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.146766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea604-2d9e-49e2-9458-fb5d571fdc16 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.147078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea604-2d9e-49e2-9458-fb5d571fdc16 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.147377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea604-2d9e-49e2-9458-fb5d571fdc16 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.151072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 382/748] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:23.151072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea604-2d9e-49e2-9458-fb5d571fdc16 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.151870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc4ed-900a-4a4b-8e5b-36d44b18dbbe None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.190241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.190935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.191217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.191489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.192210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.192639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.194000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.210054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 367/749] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.210054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.258412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea604-2d9e-49e2-9458-fb5d571fdc16 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.289199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.291910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.292846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.293141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.293395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.304572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.306007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.320785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f30dd0-20cc-4190-80d4-82c0dd74e720 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-61ea9033-10cb-45d8-8773-01b35c5b0bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:23.322992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:23.323898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:23.324790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:23.324790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:23.324790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:23.324790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:23.324790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 383/750] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:23.325926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.325926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.325926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.343511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:23.344744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:23.355513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.362350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.363167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.363451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.363721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.375025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.376634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:23.394348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:23.395317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:23.396226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:23.396226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:23.396226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:23.396226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:23.396226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 368/751] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:23.396519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.421989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 384/752] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:23.422857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.520355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb51282-1111-4246-bcc6-fd4418409e43 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.540055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e8cdf-777b-4f03-9b36-105c33b4b396 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.552014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.563652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.564445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.564724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.565000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.571820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.573270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.576457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.577310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.577800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.578253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.579232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.579611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.581030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:23.588089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:23.589089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 369/753] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:23.590014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.597693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 385/754] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.598497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.604012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.696318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e10016-5569-4cda-8972-590739b5f744 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.703983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d254db-d95c-4b53-af1d-caf0c7ba15c5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.736997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.737689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.737971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.738255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.738990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.739300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.740671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.745203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.746004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.746279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.746547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.747237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.748684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.758427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 370/755] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.758427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:23.761874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:23.762830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:23.763681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 386/756] 10.222.0.33 () {56 vars in 1129 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:23.764525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.815174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.876310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e16843-c7e7-4e23-a567-7b96e6c279f5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:23.916373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:23.917295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:23.917576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:23.917852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:23.918640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:23.919224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:23.921536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:23.935396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 371/757] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:23.936440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:23.961838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:23.997204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e0ee-7fc7-41c5-af9e-50f62bdada4c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.035082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.035959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.036339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.036615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.040003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c763f-c8d7-4253-9621-f38099b75a26 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.043674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.045107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.047127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:24.061635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:24.062545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:24.063529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 387/758] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:24.064027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.081866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.083780 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.084295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.085625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.094200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.096487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:24.117526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:24.118525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:24.119536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:24.120088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 372/759] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:24.120945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.184544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062fa03-f0e9-4bff-aedd-fb5e398e9844 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.211654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.220001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.220664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.220944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.221217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.221926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.222229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.223617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.228421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 388/760] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.229237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.229934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d55a2-4e31-44c1-9bfb-03e36c2975ee None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.267993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.268643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.268927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.269202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.270031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=01bd308fead745c7995c1bcf554cec1f) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.270343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.271813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.277300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 373/761] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/role_assignments?user.id=01bd308fead745c7995c1bcf554cec1f&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.278070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.323385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.371214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8f3a-76e6-4259-8c9a-b98d81d46c0c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.414110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.414842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.415138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.415414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.416244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.416557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.418069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.434073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 389/762] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.435075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.511905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.530593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ed8c-d2ba-4cec-b3d4-0d51b5d304f1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.565743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382ac3-22c7-4977-a73b-b61a286be1e7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.566111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.566803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.567088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.567362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.568198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.568502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.569974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.594822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 374/763] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.595981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.605290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.606320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.606852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.607330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.608288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.608840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.610476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.617019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 390/764] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.618142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.687571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.700185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5982e-f533-4ee8-849f-9e4680c70208 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.729573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7de656-96d9-4755-88bb-af6502de0573 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.735837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.736803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.737119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.737402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.739560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=42f57540b6cd47559628031b441a1eb9, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.741653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.755921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.760021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 375/765] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:24 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:24.761174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.781713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.782889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.783427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.783967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.864490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.866068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa606b-fae3-4815-9470-b4faba365190 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.886608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a0983-b4fa-4515-9945-ff4e8c8e9054 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:24.939573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:24.940160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.940439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.940710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.941519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] RBAC: Adding query filter params (interface=public, service_id=d203ab213c0c442f90570b8d888b5c1f, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:24.941828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:24.943276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0586fa-5789-4381-b89e-6c9c1ac5a07c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:24.953041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 376/766] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/endpoints?service_id=d203ab213c0c442f90570b8d888b5c1f&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:24.954551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030898d-82fb-464f-8e42-9abaaaf9d02e None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.955428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030898d-82fb-464f-8e42-9abaaaf9d02e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.955428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030898d-82fb-464f-8e42-9abaaaf9d02e None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:24.961085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 377/767] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:24.961085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030898d-82fb-464f-8e42-9abaaaf9d02e None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:24.978432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:24.994670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 391/768] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:24.996541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:24.996824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:24.997096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.018674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:25.020233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:25.032350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.087072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030898d-82fb-464f-8e42-9abaaaf9d02e None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.108276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 392/769] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:25.109378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.146574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.147829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.148119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.148396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.149086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.150471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.156800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 378/770] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:24 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:25.157680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.281506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e61309-c0c5-4d46-8d51-72f327a16b96 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.318165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.359913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.360713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.360996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.361268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.362022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2764c17-fbce-4743-aed2-8aceb9efb543 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.372885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.374462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.382960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:25.395849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:25.396791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:25.402851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:25.402851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:25.402851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:25.402851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:25.402851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 393/771] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:25.405018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.411581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.412308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.412596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.412900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] PATH_INFO: `/v3/services/network` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.413620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.415095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:25.430402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:25.431498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:25.432695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 379/772] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:25.438751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.546754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc75a3-61c8-495a-95dc-cc7fb4fa35d2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.595181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.595914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.596324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.596566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.597577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:25.597705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.599095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.607477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a3b2e-de2e-4bd0-91b1-ade6374b42ff None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.629954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 394/773] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:25.636243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.654547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.655615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.662802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.663975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.664864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] RBAC: Adding query filter params (name=network) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:25.665323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.667099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.672445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 380/774] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:25.679819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.690070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.760386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.792031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd332-6260-437f-880a-6d28abd6f9da None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.831061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.831620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.831911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.832233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:25.832944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:25.833305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:25.835278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:25.841181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 381/775] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:25.846636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:25.917012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686508a-41e8-49c1-8774-3be60a659ebf None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.919345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:25.957020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e1c0-1282-4697-b5f0-e4ee2804dff5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:25.982808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:25.985429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:25.985710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:25.985985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.000852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:26.002307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:26.026995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:26.027652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.027933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.028209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.029002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] RBAC: Adding query filter params (interface=public, service_id=f45d0c696cea4b85acb0f40e7ac00078, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:26.029311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:26.030692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:26.034435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:26.035423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:26.036326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:26.037223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 395/776] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:26.040775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 382/777] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:25 2022] GET /identity/v3/endpoints?service_id=f45d0c696cea4b85acb0f40e7ac00078&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:26.042676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:26.046360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:26.233174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a80d7-e4c7-4f5b-97a8-0432dd73ee02 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:26.240868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b490b-d4a3-4460-b6c7-f28214492082 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:26.251195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:26.280149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5297b-f075-4081-a123-a818505975db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:26.280811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5297b-f075-4081-a123-a818505975db None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.281089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5297b-f075-4081-a123-a818505975db None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.281359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5297b-f075-4081-a123-a818505975db None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.282155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5297b-f075-4081-a123-a818505975db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:26.282461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5297b-f075-4081-a123-a818505975db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:26.283939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5297b-f075-4081-a123-a818505975db None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:26.307136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 396/778] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:26.307321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:26.309406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:26.310973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.310973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.310973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.311313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:26.313076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c22eb-b777-4d8c-b433-712940133040 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:26.315075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5297b-f075-4081-a123-a818505975db None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:26.327303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 383/779] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:26 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:26.426065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5297b-f075-4081-a123-a818505975db None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:26.464928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:26.465602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.465882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.466153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.467039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2eb34259fb6a4963b9277d2607c21468) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:26.467357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:26.468818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271c982-e695-4630-8ba7-bc92b2fe19fe None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:26.478824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 397/780] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:26 2022] GET /identity/v3/role_assignments?user.id=2eb34259fb6a4963b9277d2607c21468&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:26.771078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23755de-fd9b-4b9f-ab3f-a35e5657cf01 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.771559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23755de-fd9b-4b9f-ab3f-a35e5657cf01 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.771559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23755de-fd9b-4b9f-ab3f-a35e5657cf01 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.773470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 384/781] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:26.779214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:26.779518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:26.779783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:26.805157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:26.806365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:26.851332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:26.906108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 398/782] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:26.942548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23755de-fd9b-4b9f-ab3f-a35e5657cf01 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.122421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23755de-fd9b-4b9f-ab3f-a35e5657cf01 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.183878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:27.185056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.185573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.186037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.209936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-10b86a42-d54f-4def-85b7-0e44577d3535 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:27.218107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:27.219070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:27.219937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:27.231306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 385/783] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:27.232196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.327797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb061-892e-48a2-bd9d-b4ba6f85aad8 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.328451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb061-892e-48a2-bd9d-b4ba6f85aad8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.328932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cb061-892e-48a2-bd9d-b4ba6f85aad8 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.331795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 386/784] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:27.335945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.336447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.336915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.354435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:27.355999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:27.400926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9179d-12aa-4da7-a703-09b4db76a256 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.459173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:27.463137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:27.463809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.464090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.464362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.465115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:27.465420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:27.466805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:27.487240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 399/785] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:27.488757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 387/786] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:27.499316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.516306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.579133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:27.687271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757b2d3-8a99-4db0-a0b1-da6714930e5b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.706823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33afc4-720c-48ba-8553-a2c838925366 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.756585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:27.757381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.757662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.757934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.774389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:27.775458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.775758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.776140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.776855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:27.785733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:27.786759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:27.787632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:27.794917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 388/787] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:27.797620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.813176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:27.815094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6d71015b-202e-460c-8da7-184d01ad9cb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:27.847637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:27.848953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:27.849867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:27.849867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:27.849867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:27.849867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:27.851005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 400/788] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:27.851963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde0ed8-7692-4876-b218-5087518f3e7d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.852524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde0ed8-7692-4876-b218-5087518f3e7d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.852781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde0ed8-7692-4876-b218-5087518f3e7d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.858904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 401/789] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:27.859999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde0ed8-7692-4876-b218-5087518f3e7d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:27.875505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:27.942845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afe8c-c690-455c-a408-4feb74a3ccef None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.957574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:27.976863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde0ed8-7692-4876-b218-5087518f3e7d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:27.988090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:27.988789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:27.989126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:27.989402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:27.990178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:27.991073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:27.992912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a64c60-86d5-4948-887b-e8e3517bfbae None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.006888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 389/790] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:28.008546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.009039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.009538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.027887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.028946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.029424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.029706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.030463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:28.030796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.032247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.036190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:28.037792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:28.050578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 402/791] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:28.051508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.114225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 390/792] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:28.115226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.156768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.177768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b228c-7cbf-4055-a48c-c64021714f46 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.213268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.214569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.214890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.215170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.222397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1485f-3842-43ad-b581-27bfddf81d1c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.228627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.230077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.239191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:28.253573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:28.254666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:28.255795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:28.255795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:28.255795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:28.255795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:28.258746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 403/793] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:28.259943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.264764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.265756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.266198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.266614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.275058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.276755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:28.292645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:28.293792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:28.294879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:28.297264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 391/794] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:28.298298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.372844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2443b0a-e469-499d-a4be-467216e93c21 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.410609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.411845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.412308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.412787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.413724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.415417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.416525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.420556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af7593-a4fc-4c22-aecb-f04583bf02fa None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:28.431288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:28.432236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 00:22:28.433061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:28.436196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 404/795] 10.222.0.33 () {56 vars in 1129 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:28.437191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.468299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.468880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.469158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.469427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.470145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:28.470451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.471866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.505732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 392/796] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:28.505732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.527393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.569601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ca65-f0a7-456b-9744-a8bdc1cbd6ef None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.606106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.606857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.607133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.607402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.608254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:28.608614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.610046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.633502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 405/797] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:28.633502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.646817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401e461-b59a-4690-b3e6-0bf5f76622cc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.683016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.683881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.684258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.684630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.685436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:28.685863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.687377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.697719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 393/798] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:28.697719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.720969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.774876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4d84a-6caa-4dcb-863e-f2fd3bdf53f3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.797707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:28.810210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a101cf-e376-4565-b0ad-99e165955371 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.814382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.815406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.815825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.816263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.824603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.826213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:28.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:28.842064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:28.843467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:28.850521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 406/799] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:28.850521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.851152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:28.851938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:28.852479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:28.852747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:28.864655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:28.866652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:28.885089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:28.886011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:28.886911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:28.886911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:28.886911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:28.886911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:28.890917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 394/800] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:28.891937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:28.961809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c967c0-5426-4926-bf25-aa29253f1a02 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:28.970096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.001800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042a37-7022-4b77-b625-8eb546447b4c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.002530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.003483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.003965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.004436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.005410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:29.005935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.007689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.018686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 407/801] 10.222.0.33 () {56 vars in 1124 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:29.019971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.041289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.042040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.042320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.042642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.043775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:29.043921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.045455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.070387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 395/802] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:29.071571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.074737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.130899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01565-0aa2-4d76-9651-7d4c08f8539c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.163344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.163991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.164265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.164530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.165239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:29.165538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.166914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbacc2c-06ab-46cf-806b-21b9e1fc4024 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.172028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b249c-4e7a-475d-90fe-d288f228c39c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.177776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 408/803] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:29.179646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9fe7-156d-4b81-ba09-a85991db6ca4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.179934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9fe7-156d-4b81-ba09-a85991db6ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.180235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9fe7-156d-4b81-ba09-a85991db6ca4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.181405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 409/804] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:29.182088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb9fe7-156d-4b81-ba09-a85991db6ca4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.204591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.209672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.210315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.210593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.210897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.211629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.212985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38270f98-5a10-4dcc-a16d-696c74156903 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.233902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 396/805] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:29.235252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689df8cd-8e89-4c36-a0fc-27f41cb55520 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.235537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689df8cd-8e89-4c36-a0fc-27f41cb55520 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.235810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689df8cd-8e89-4c36-a0fc-27f41cb55520 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.237054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 397/806] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:29.237865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689df8cd-8e89-4c36-a0fc-27f41cb55520 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.304398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.309650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb9fe7-156d-4b81-ba09-a85991db6ca4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.345406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.346055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.346330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.346603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.351176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689df8cd-8e89-4c36-a0fc-27f41cb55520 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.389137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.389927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.390209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.390477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.398521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.399994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-f54aae07-cbce-4d5b-b8f0-de1be5a048e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:29.414238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:29.415201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:29.416234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:29.420119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 398/807] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:29.421398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53349a-ff8a-4bb9-b2a4-120e4d116f7c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.422335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53349a-ff8a-4bb9-b2a4-120e4d116f7c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.422335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53349a-ff8a-4bb9-b2a4-120e4d116f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.428553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.430021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.439912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5e53349a-ff8a-4bb9-b2a4-120e4d116f7c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:29.441101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5e53349a-ff8a-4bb9-b2a4-120e4d116f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:29.497532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.534798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 399/808] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:29.534798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.534798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.534798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.536132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.536593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 410/809] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:29.537453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.555434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:29.556959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:29.631769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 400/810] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:29.639895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.643078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b211-acc0-41a8-b87c-460eeaf67cdd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.679647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.681422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.681929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.682396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.683415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:29.683939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.685961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.704791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 411/811] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:29.705932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.746569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2988d-9e1e-464a-b699-487c09fbc8cc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.756228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.773415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:29.783033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.783823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.784105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.784374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.795643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.797088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:29.820518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:29.821857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 401/812] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:29.822861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.823644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315641b7-eefd-42d2-b82e-2827fbf22ce0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.863094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.863876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.864149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.864416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.874973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:29.883182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:29.884128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:29.884953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 412/813] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:29.885642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.936762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095175f-8e9f-4db3-8d1c-aac821fa2eff None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:29.972597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:29.973377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:29.973654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:29.973925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] PATH_INFO: `/v3/services/network` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:29.974573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:29.975996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:29.987858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:29.988857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:29.989708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 402/814] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:29.990526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:29.997310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddef84-5978-4ab0-8615-db8184c0fc94 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.020387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.021049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.037541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.038467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.038739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.039044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/01bd308fead745c7995c1bcf554cec1f/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.040996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=01bd308fead745c7995c1bcf554cec1f, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.042415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.056504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 413/815] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:29 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/01bd308fead745c7995c1bcf554cec1f/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:30.057645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.136706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea005459-bca0-4c6a-b74d-8ea5d394eb1b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.182226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d09550-708a-462d-8512-fc8d9403737e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.186469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.187174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.187458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.187735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.188471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.188777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.190178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.213155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 403/816] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.213964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.230295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.231102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.231296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.231659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.232303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.232604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.234045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.243968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 414/817] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.244786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.301638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.315829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.336506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4e8f-e0f2-410f-869f-086dbe21058c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.373679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a42e1-16c0-4de7-b502-39335324b63b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.378740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.379449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.379725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.379991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.380705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] RBAC: Adding query filter params (name=network) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.381005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.383180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.394116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 404/818] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.394116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.416769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.417999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.418475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.418971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.426680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.429079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:30.444972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:30.445887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:30.446767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:30.449372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 415/819] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:30.450575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.512799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af169a84-0cd6-4c2b-ac21-01667b94a93e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.551685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.552628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.552914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.553331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.565192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.566838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.573557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce5f85-ccc1-4f62-9bee-68d47e4bc29b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.575756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.577690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:30.589679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:30.590609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:30.591605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:30.591605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:30.591605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:30.591605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:30.591605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 405/820] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:30.591891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.617744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.618393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.618671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.618980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.619679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] RBAC: Adding query filter params (type=network) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.619999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.621338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.626789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 416/821] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.627594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.733887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef4569-7751-4622-a918-a6c75e576b92 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.735743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacecef0-787e-46b9-a121-0e6704592247 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.771027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.771680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.771980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.772273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.773093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.773398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.774954 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.783473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.785115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.785719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.786200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.787182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:30.788207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 406/822] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.788452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.789010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.790244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.806945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 417/823] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:30.808309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:30.852260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.857688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:30.902163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8546d43-b703-4174-b219-c6d0a9fb817f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.946035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:30.947082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:30.947451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce107-0142-48fc-88f9-a639337c68e8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:30.947812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:30.948282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:30.949501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:30.951963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:30.978440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 407/824] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:30.979653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.007887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.008526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.008848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.009124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.010016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:31.010420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.011808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.025927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 418/825] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:31.111890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd87d2-40fb-44eb-af72-05a1804ef490 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.133092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.158756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.159982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.160484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.160985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.171014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.172861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:31.189125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:31.190135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:31.191114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:31.193906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 408/826] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:31.199460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.300639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.317013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e5a2a-cd1d-4563-917d-90f141710da1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.355822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.356492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.356770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.357294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.358208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:31.358474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.360081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673280c0-a855-484e-97f1-ad1625756e72 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.382314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 419/827] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:31.387961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.437804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d0f4-3cd8-4375-9629-496ae28b10d2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.438118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d0f4-3cd8-4375-9629-496ae28b10d2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.438356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d0f4-3cd8-4375-9629-496ae28b10d2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.442798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 420/828] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:31.445362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741975-5fb5-4f00-803f-831011fef720 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.445646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741975-5fb5-4f00-803f-831011fef720 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.445920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741975-5fb5-4f00-803f-831011fef720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.463890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-11741975-5fb5-4f00-803f-831011fef720 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:31.465088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-11741975-5fb5-4f00-803f-831011fef720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:31.503320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492c8cc-079f-472f-b43c-df6838b383c7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.542867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 421/829] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:31.548372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.549629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.550315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.551037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.562074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.563640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741975-5fb5-4f00-803f-831011fef720 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.568221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:31.589269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:31.590200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:31.591528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:31.591528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:31.591528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:31.591528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:31.593578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 409/830] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:31.602928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.643832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.659138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.669883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741975-5fb5-4f00-803f-831011fef720 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.706461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.707480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.707762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.708031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.708697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.710345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.714241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a4c5e-ebec-42d7-95fb-394fc7e95ea6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:31.722432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:31.723442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:31.724809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 422/831] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:31.730926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.755536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.757226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.757636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.758048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.758999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:31.759459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.761221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.778224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 410/832] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:31.779829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.841191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caf82-be7a-447e-b5cd-66424856494a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.874895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.882766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.883714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.884194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.884792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.885893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:31.886407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.888001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.888230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee02d55-11d5-4d33-a0e8-2a8857e57f06 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:31.893887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 423/833] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:31.901726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:31.901885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:31.925449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:31.926352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:31.926634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:31.926947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:31.934966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:31.936395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:31.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:31.953581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:31.954500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:31.958708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 411/834] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:31.959953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:32.016654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c35c3-abcb-4c0e-bec9-ea43455d7a2e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:32.052041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:32.052708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.052983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.053833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.054547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:32.054885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:32.056281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:32.067006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 424/835] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:32.092270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146510-126e-4b02-8f0b-ed7d09851177 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:32.132123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:32.132794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.133071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.133341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.134136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:32.134441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:32.135944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b07a-6fb1-47f7-b06d-714eeb4f4eba None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:32.153689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 412/836] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:32.155116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:32.226800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:32.265079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894331be-9d15-48d2-a4f0-b5002a94e680 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:32.300612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:32.301501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.301774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.302037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2eb34259fb6a4963b9277d2607c21468/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.317992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=2eb34259fb6a4963b9277d2607c21468, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:32.319555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bbc25-0604-42d9-9103-e2bbdfe45956 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:32.343018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 425/837] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:32 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2eb34259fb6a4963b9277d2607c21468/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:32.429658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4789dc-9282-4c4d-9b65-8f1745bada09 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.429994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4789dc-9282-4c4d-9b65-8f1745bada09 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.430212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4789dc-9282-4c4d-9b65-8f1745bada09 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.431983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 413/838] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:32.437773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.438109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.438334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.455803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:32.457014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:32.544257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 426/839] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:32.569968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4789dc-9282-4c4d-9b65-8f1745bada09 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:32.581735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:32.677488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4789dc-9282-4c4d-9b65-8f1745bada09 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:32.712057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:32.712845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.713126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.713395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.723819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:32.732372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:32.733302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:32.734167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 414/840] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:32.739435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:32.787101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:32.849041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8cd0f-522b-463f-9f17-595c94893050 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:32.886670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:32.887367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:32.887647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:32.887923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:32.888677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:32.888981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:32.890345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:32.899967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 427/841] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:32.905332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:32.996073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.016539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e803d1-e34e-4c92-b1ed-532034c1e148 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.051524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.052360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.052640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.052913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.064542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.065998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:33.083469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:33.086091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:33.088261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:33.088261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:33.088261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:33.088261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:33.088261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 415/842] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:33.092011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.201695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf6d35-0916-4183-a734-737578058a67 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.211841 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.239774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.240843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.241465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.241922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.242886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:33.243547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.245143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.259127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 428/843] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:33.267899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.374824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8eb71f-7abf-4ec1-9159-a46ba259abbe None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.399051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.415984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.416800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.417083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.417359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.428363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.429931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c9883eda-fe0f-42f8-8fd6-2de6d809c6a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:33.445439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:33.446419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:33.447397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 416/844] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:33.455833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.463083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b24fd-b450-4dcf-a147-f7635eb13284 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.463368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b24fd-b450-4dcf-a147-f7635eb13284 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.463648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b24fd-b450-4dcf-a147-f7635eb13284 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.465256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 417/845] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:33.470536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.470852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.471136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.490102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:33.491458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:33.566759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e3065-eee3-4449-9470-9b5e48015215 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.570915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 418/846] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:33.591379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.608173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.609143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.609697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.610164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.611293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:33.611860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.613628 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.633409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 429/847] 10.222.0.33 () {56 vars in 1135 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:33.638816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.677208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.697545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646abd-2dab-4fdb-96e0-88f2e2a3e7d1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.708890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.734267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.735291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.735570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.735841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.736494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.737857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:33.751165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:33.752136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:33.752990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 419/848] 10.222.0.33 () {56 vars in 1132 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:33.758657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.774205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb807bb7-7d5d-4951-8b73-6bd906d67211 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.812770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.813869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.814353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.814902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.872156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816a5f-3c5b-4987-9878-a2f243434520 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:33.888295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.890016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.907033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:33.907716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:33.907963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:33.908234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:33.908926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:33.909225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:33.910661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:33.919968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 420/849] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:33.920980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:33.925322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:33.967251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.004889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 430/850] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:34.011481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.041027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690246af-4703-452e-8ec8-aeab2ce90c0d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.086099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.087242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.087861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.088447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.089352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:34.089852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.091460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.097250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 421/851] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:34.125800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c98714-9564-41ce-b3af-a3eba17c5005 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.157911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.168836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.170029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.170510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.171046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.183597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.185433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7a157cb0-4e3d-478a-b188-e127c98f3df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:34.206505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:34.207493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:34.208519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:34.208519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:34.208519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:34.208519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:34.215072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 431/852] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:34.217174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.337048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1091598-5d4c-464a-917a-2fbb570050ec None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.339127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.348592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992e7e1-a012-4bd7-8dba-e49e6f2d3803 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.348886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992e7e1-a012-4bd7-8dba-e49e6f2d3803 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.349149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992e7e1-a012-4bd7-8dba-e49e6f2d3803 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.351348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 432/853] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:34.356631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.356917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.357193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.376267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:34.377402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:34.386337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.387301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.388170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.388644 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.390030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:34.390614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.392373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.413684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 422/854] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:34.415432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.459994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 433/855] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:34.477797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.544879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc606-df15-466c-a7af-ec131581dfa1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.588011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ba938-5ef0-49a2-adfe-f0f463a8555d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.599714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.600217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.601540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.601679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.602253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.609702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.611179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.626446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.627079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.627356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-03313cb5-7b33-462c-b08b-e5d3c8f3ce81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:34.627453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:34.628281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:34.629196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:34.629627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.629627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.630249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aa588-8a13-4009-a6c7-9ca33c4534c2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.634662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 423/856] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:34.635938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d587-99db-4284-93af-fc8422fad9cc None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.636258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d587-99db-4284-93af-fc8422fad9cc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.636585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d587-99db-4284-93af-fc8422fad9cc None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.637978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 424/857] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:34.639537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283d587-99db-4284-93af-fc8422fad9cc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.651042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 434/858] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:34 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:22:34.651705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.651705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.651919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.668286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.684177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:34.686533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:34.778442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283d587-99db-4284-93af-fc8422fad9cc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.786929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 435/859] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:34.816088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.827558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:34.828523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:34.828817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:34.829090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:34.830319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:34.830633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:34.832656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:34.848578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 425/860] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:34.854883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:34.947858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.954348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6219d29-9ab0-4109-8ae0-6282cbd0b368 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.960277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:34.989632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749df1-7e6f-467b-abf8-8f1aa4bb73ce None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:34.999465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.000262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.000538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.000807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.011516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:35.019345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:35.020286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:35.021113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 436/861] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:35.027294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.039817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.040527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.040813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.041089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.042295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=01bd308fead745c7995c1bcf554cec1f) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:35.042663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.044794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.052742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 426/862] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:34 2022] GET /identity/v3/role_assignments?user.id=01bd308fead745c7995c1bcf554cec1f&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:35.155062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c590c-90b2-4bb7-afd9-f738969709d4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:35.196193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.196879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.197157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.197431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.198256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:35.198504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.199976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.210067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 437/863] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:35.215648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.325457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305652-de63-4c5e-bb1e-8d5d6f3292e2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:35.360443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.361234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.361516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.361789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.373576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.375067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.376999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:35.394854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:35.396996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:35.399173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:35.399173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:35.399173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:35.399173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:35.401698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.402148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 427/864] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:35.424917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:35.514615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a3f6e-4b58-4196-8ba1-c30d65b9a6b5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:35.548491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.549150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.549438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.549711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.550437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:35.550741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.552127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.565239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 438/865] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:35.571415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.673258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c1f4a-b2fe-47f2-8ffb-4c9a87477e1d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:35.708151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.708967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.709282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.709557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.718122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.719604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:35.735033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:35.736005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:35.736949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 428/866] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:35.743374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.828760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:35.848538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c38603-bc47-4084-8c50-c6121947dc87 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:35.886811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:35.887482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:35.887761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:35.888031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:35.888874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:35.889178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:35.890604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:35.908779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 439/867] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:35.914848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:35.975813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:36.018610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8139f9f-7415-4b62-9783-8ef65e354f65 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.054004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.054650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.054960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.055233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.128342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.129713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd705f-877d-4a6e-8cdf-e95b43def633 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:36.191135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:36.231508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 429/868] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:35 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:36.237545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:36.321894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c3c6e-758c-42bb-b627-96e9c0c6f2bd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.322247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c3c6e-758c-42bb-b627-96e9c0c6f2bd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.322450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c3c6e-758c-42bb-b627-96e9c0c6f2bd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.324109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 430/869] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:36.331287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.331685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.332059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.354183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e27c4-848d-4233-bddf-80b01b521481 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.355115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:36.356658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:36.392603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.393675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.394267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.394691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.403090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:36.408166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.410248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:36.430132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:36.431259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:36.432162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:36.432162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:36.432162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:36.432162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:36.434403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 440/870] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:36.440452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:36.450824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 431/871] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:36.468409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:36.553793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d20561-70c0-46ae-a406-1d9a985bfd06 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.561196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:36.585450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6cf5f-d1b5-42be-a7eb-408d73c318d3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.593743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.594404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.594690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.595402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.595826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:36.596136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.597825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:36.615039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 441/872] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:36.616095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:36.629274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.630590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.631934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.632410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.633598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.635956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:36.662882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 432/873] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:36 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:22:36.691909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:36.746979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b38f1-a6f5-4a78-8747-2edcc7e2079e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.795387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.796595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.797089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.797554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.805789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.807642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-450671c1-617c-422d-8e96-7b95b6807403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:36.824894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:36.825853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:36.826758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:36.829541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 442/874] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:36.835832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:36.936777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11dc4-f2ac-4216-88f9-b6f9f73baada None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.937284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11dc4-f2ac-4216-88f9-b6f9f73baada None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.937556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11dc4-f2ac-4216-88f9-b6f9f73baada None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.939197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 443/875] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:36.944756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.945034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.945306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.952981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ab9e0-2af2-4965-9a6a-7a31a0da4ea1 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:36.963800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:36.964974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:36.987634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:36.988442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:36.988732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:36.989008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:36.989849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:36.990166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:36.992173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.005646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 433/876] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:37.012099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.023130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.042852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 444/877] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:37.060750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.097211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.118808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f063-5d44-4c56-83ff-9d1e67f25a53 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.153275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.153933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.154219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.154502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.155385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2eb34259fb6a4963b9277d2607c21468) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:37.155704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.157536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.164580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 434/878] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/role_assignments?user.id=2eb34259fb6a4963b9277d2607c21468&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:37.171563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d79755-b045-4e6c-a9dd-37ed3027c978 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.212859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.213981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.214440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.214914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.215764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.217362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f15bd4f9-5ba0-4c05-9d31-d3a2da9295dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:37.231542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:37.232607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:37.233445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.236277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 445/879] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:37.243736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.319984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.323766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3a79-52aa-4bdd-86c2-2f1a33e3de83 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.324521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3a79-52aa-4bdd-86c2-2f1a33e3de83 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.324996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d3a79-52aa-4bdd-86c2-2f1a33e3de83 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.327649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 446/880] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:37.332238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.332516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.332785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.353984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:37.355242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:37.365418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255ee98-deb1-452e-a5cb-9dde5c463dde None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.401315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.402339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.402859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.403358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.404283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:37.404788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.406356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.411918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 435/881] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:37.418484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.443608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 447/882] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:37.466005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.504081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.545815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755e01-3f0c-4e09-98ea-af2aec0a886b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.579640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.586620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.587558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.588043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.588513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.589419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:37.589952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.591538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.593721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c39d00-e0a5-4bd4-bff9-1563d74ddda3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.597417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 436/883] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:37.604119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.635785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.636579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.636860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.637145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.637929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.639918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:37.655311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:37.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:37.657240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:37.661882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 448/884] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:37.664040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.718212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e759f6-28f3-4a80-a7c4-d4ec164a7b8a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.757532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.758282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.758578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.758860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.759652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] RBAC: Adding query filter params (interface=public, service_id=d3b286c41a954da9b32cb4039bdec4fb, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:37.759958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.761323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.773692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 437/885] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/endpoints?service_id=d3b286c41a954da9b32cb4039bdec4fb&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:37.775181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.785086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.794067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b66496-9cee-44be-9c70-de1003d17663 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.840540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.842083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.842556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.843046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.843962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:37.844459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.846017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.852589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 449/886] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:37.858378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:37.863534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:37.913837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b552ea-a120-440d-93fd-58083aa337f7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.960467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:37.961873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:37.962592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:37.963072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:37.964215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:37.966430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5df73b-b805-4e35-ac13-5eb6b7a981a2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:37.970530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7650bc-bf0c-4b5f-9617-e1e3d908ce0d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:37.983442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 438/887] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services => generated 1268 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:22:38.014424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:38.015128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:38.015463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:38.015726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:38.016498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:38.016805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:38.018190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec17aa-9417-4f10-b33c-1afd5f8e32c4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:38.024655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 450/888] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:38.884886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e42056-3126-4c56-8249-941030cfff72 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:38.885400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e42056-3126-4c56-8249-941030cfff72 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:38.885400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e42056-3126-4c56-8249-941030cfff72 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:38.886948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 439/889] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:38.892156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:38.892568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:38.892742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:38.909551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:38.910717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:38.928677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:38.981051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 451/890] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:39.003784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e42056-3126-4c56-8249-941030cfff72 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.117149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e42056-3126-4c56-8249-941030cfff72 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.146271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:39.158445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.159602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.160100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.160585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.161444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:39.163591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:39.181526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:39.183437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:39.184811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:39.187908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 440/891] 10.222.0.33 () {56 vars in 1132 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:39.195927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.302001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91885004-1cd5-4e17-8df4-82e4bd17e5b9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.337952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.338995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.339492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.339976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.340897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:39.341402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:39.343009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895179e2-74d0-4df7-a08d-8f64792a97e6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:39.348391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 452/892] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:39.355919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.457605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da74ef-4e5d-4656-8dc0-f7d6fc09702d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.504291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.504963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.505246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.505520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.506765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:39.507097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:39.508527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:39.513959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 441/893] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:39.516902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cff2b-1a5b-4c54-9e4f-e8d74692ad61 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.517190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cff2b-1a5b-4c54-9e4f-e8d74692ad61 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.517463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cff2b-1a5b-4c54-9e4f-e8d74692ad61 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.519293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 453/894] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:39.520275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.527600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.527889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.528166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.546575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:39.547711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:39.559883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:39.591935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:39.624856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 454/895] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:39.656037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.657378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee134d1-3c09-4d56-8726-030a9d2d9145 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.694352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.695430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.695917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.696412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.697422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] RBAC: Adding query filter params (interface=public, service_id=9809f60565fc414c9bcd9f16cd45fa9f, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:39.697941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:39.699547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:39.705608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 442/896] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/endpoints?service_id=9809f60565fc414c9bcd9f16cd45fa9f&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:39.710884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.779736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c528d-99ec-4986-ba61-653c02480ec3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.817067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.817876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.818154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.818450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.830118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:39.835202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231cf-040b-4f3a-9cee-9d8d9b62673c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:39.839154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:39.840093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:39.840945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 455/897] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:39.846989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:39.857234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:39.861107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:39.878729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:39.879608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:39.879984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:39.880355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:39.881729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:39.883474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:39.897178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 443/898] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:22:39.979670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a5aaa-98ec-4d35-9c70-a3dbdfbf006b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.017247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.018196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.018638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.019093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.020009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:40.020469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.022581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b74b5c-0283-4f4b-bf4d-1f1dfe6112bc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.036759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 456/899] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:22:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:40.038982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.149313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd3b6b-0366-4d53-8ee7-f2497adbee51 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.186083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.187219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.187644 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.188072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.199792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58773-9c3a-499a-a79c-ce0539340328 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.200436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58773-9c3a-499a-a79c-ce0539340328 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.200908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58773-9c3a-499a-a79c-ce0539340328 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.202269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.204241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.204496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 457/900] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:40.209685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d85a-3fe2-40ec-a578-cd4b253e6a2c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.210100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d85a-3fe2-40ec-a578-cd4b253e6a2c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.210495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d85a-3fe2-40ec-a578-cd4b253e6a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:40.225964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:40.227478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:40.228578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:40.228578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:40.228578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:40.228578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:40.230552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 444/901] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:40.231972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0ee8d85a-3fe2-40ec-a578-cd4b253e6a2c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:40.233424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ee8d85a-3fe2-40ec-a578-cd4b253e6a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:40.236247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.270555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.302854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.310955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 458/902] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:40.312595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737023c6-75bc-4c1c-b9aa-4621da28da13 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.313109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737023c6-75bc-4c1c-b9aa-4621da28da13 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.313588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737023c6-75bc-4c1c-b9aa-4621da28da13 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.319721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 459/903] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:40.321095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.321592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.322063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.340493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:40.341673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:40.350453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18122f77-15c4-46b1-8097-2ffb0addf28d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.386726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.387433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.387711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.387982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.388691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:40.388993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.390410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.403616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 445/904] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:40.404394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.418796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 460/905] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:40.419955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.515125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.525548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c705-5bd6-4f48-ba78-f1d0e7bb2cab None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.539523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cb60f-2dc4-45e5-9246-adb77f2c1439 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.563017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.563676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.563950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.564218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.564933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.566280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.568012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.578319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.579153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.579429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.579705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.580250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 446/906] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:40 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:40.581038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.597856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.600597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:40.619117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:40.620171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:40.621128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:40.628278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 461/907] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:40.629574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.733322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ce2d8-8d8e-4fea-be14-c56644e16dcd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.753666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d63fe8-6b19-4b75-a789-5a9d1ab8b55b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.782578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.783455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.783742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.784022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.784683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.786064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.797607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.798478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.798965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.799398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:40.800028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:40.801194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:40.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 447/908] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:40.804584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:40.804905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.806344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.808653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.825538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.826485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 462/909] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:40.832989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:40.914411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:40.935608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0b9c0-dd8c-460c-8dac-7e3d9aafed56 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.979777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef72cd8-6c2d-4d4a-b0cb-7a4dbc65053f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:40.981421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:40.982455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:40.982861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:40.984140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:40.984969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:40.985382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:40.986902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:40.994064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 448/910] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:40.999252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.027230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.027829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.028114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.028388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.098110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:41.122979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.124872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.128466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfde04-721f-4731-aeb7-d672dd30efe7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.167999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.168671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.168956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.169231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.170240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:41.170362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.171755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.178406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:41.185319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 449/911] 10.222.0.33 () {56 vars in 1114 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:41.187024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.247572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 463/912] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:40 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:41.255202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.306359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf1912-4a68-4897-849d-94cf2bb9d510 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.344313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.345223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.345587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.345959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.346851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.348409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.357707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:41.359216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758e9c7-75d1-42d6-8a88-5287ff2b2047 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.374732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 450/913] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:41.412799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.413996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.414506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.415013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.427287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.429736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.432015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:41.452328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:41.453364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:41.454263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:41.454263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:41.454263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:41.454263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:41.461177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 464/914] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:41.463169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.585783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a669-ec01-4c21-a0b6-98e6ebd5e0e6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.621324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.621980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.622256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.622526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.623278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:41.623582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.624936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.638032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 451/915] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:41.643624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.748614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133408b-15af-4329-b8e7-f4a95f9526b4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.786453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.788356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.788634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.788900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.796826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.799498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:41.817452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:41.819221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:41.820255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 465/916] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:41.826732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:41.886714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:41.932019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413a1c5-66a5-4be0-a6b3-b28ff3b009f6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:41.967481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:41.968843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:41.969283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:41.969696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:41.970618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:41.971087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:41.972670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9c88c-893f-4768-adbf-4164fd64d772 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:41.989971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 452/917] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:41.991576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.049075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:42.107164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0cff5-11e7-47a7-9660-a06e601f2fed None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:42.147338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:42.147915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.148192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.148464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.149462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] RBAC: Adding query filter params (role.id=5adee2b2a8224db8aaca0778a60b9ffb, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2eb34259fb6a4963b9277d2607c21468) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:42.149779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:42.151226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:42.153037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6b7c9-5b9e-4b8b-92e0-be3ca091e5fc None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.153326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6b7c9-5b9e-4b8b-92e0-be3ca091e5fc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.153602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6b7c9-5b9e-4b8b-92e0-be3ca091e5fc None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.155260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 453/918] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:42.156969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 466/919] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:41 2022] GET /identity/v3/role_assignments?user.id=2eb34259fb6a4963b9277d2607c21468&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=5adee2b2a8224db8aaca0778a60b9ffb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:42.164326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.164652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.164875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.193121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:42.195004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:42.274534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 454/920] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:42.297763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.408036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31532246-d1e6-4051-b7c6-f2ab8cfbeb64 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:42.425458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:42.444832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:42.445649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.445928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.446201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.446887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:42.448329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:42.460891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:42.461860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:42.463146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 467/921] 10.222.0.33 () {56 vars in 1132 bytes} [Wed Jun 15 00:22:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:42.468391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.518367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:42.575972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f38067-dcac-4443-9a54-21b8592cc860 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:42.611395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:42.612103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.612386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.612710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.613417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:42.613724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:42.615153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:42.620839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 455/922] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:42.626388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.743768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ee543-ed7d-4b41-a169-522b4f392115 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:42.782155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:42.782874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.783156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.783429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.784162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:42.784469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:42.785957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b9d56-7edf-4f72-af8f-cc859c363062 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:42.792744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 468/923] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:42.798303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.829369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219c90-b595-49af-be1a-5cd72732c7c8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.829667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219c90-b595-49af-be1a-5cd72732c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.829929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219c90-b595-49af-be1a-5cd72732c7c8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.831543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 469/924] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:42.837068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.837349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.837625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.855223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:42.856395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:42.912218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ea6b3-e390-4313-8571-b5c2b15f0d26 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:42.912840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:42.926420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:42.953040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 470/925] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:42.962813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:42.962813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:42.962813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:42.962813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:42.963165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:42.963165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:42.975822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:42.990798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 456/926] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:43.092991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f30644-c519-44a3-8e94-c0e105b64768 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:43.138196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:43.139108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.139670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.139944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.140615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:43.141997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:43.154788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:43.155817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:43.156644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 471/927] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:43.163454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:43.280031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926be153-c5e4-478f-9f24-48bf6d49323f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:43.316417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:43.317604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.318097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.318594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.319523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:43.320039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:43.321587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:43.327173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 457/928] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:43.333149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:43.377184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:43.446701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a899ba1-1a3c-4124-ae8a-2a76eaf586fe None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:43.489308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:43.491003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.491502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.491965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.492874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:43.493498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:43.494819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:43.500429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 472/929] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:43.506198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:43.539410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:43.612506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061a3f-2ab7-4225-b922-5f948c91d873 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:43.647708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:43.648285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.648563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.648830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.649633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] RBAC: Adding query filter params (interface=public, service_id=c35ac621c87343a284b20d3ae747802f, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:43.649940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:43.651484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27efbc9-0764-412b-b7a0-8cec0ab942b7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:43.657981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 458/930] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:43 2022] GET /identity/v3/endpoints?service_id=c35ac621c87343a284b20d3ae747802f&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:43.664222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:43.727810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58dae8-bf70-4946-9cf7-36e33a82d46a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.728135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58dae8-bf70-4946-9cf7-36e33a82d46a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.728436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58dae8-bf70-4946-9cf7-36e33a82d46a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.730443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 459/931] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:43.739322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.739702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.740141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.761810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:43.763569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:43.776268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:43.779892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a25e1-7bf5-4460-8644-8f0efb69c21d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:43.818310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:43.819330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:43.819761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:43.820187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:43.821052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:43.822650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:43.829622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 473/932] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:43 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:22:43.856110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 460/933] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:43.878857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:43.991406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27a45-ee1d-438a-bb9f-25899d382a7c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:44.032634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:44.033290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.033567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.033835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.034589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:44.036008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:44.037402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING keystone.server.flask.common [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 00:22:44.055361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.sql.core [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 00:22:44.055361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 00:22:44.055361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [parameters: {'id': 'e3bd3d4c5c3d4c11b26facf0daf99b6c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 00:22:44.055361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3bd3d4c5c3d4c11b26facf0daf99b6c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3bd3d4c5c3d4c11b26facf0daf99b6c. Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:22:44.073362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 00:22:44.074229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application yield resource Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise self.value Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application session.commit() Jun 15 00:22:44.075697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.flush() Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application compat.raise_( Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 00:22:44.076812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 00:22:44.077699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:22:44.078563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [parameters: {'id': 'e3bd3d4c5c3d4c11b26facf0daf99b6c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.079462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:44.091500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3bd3d4c5c3d4c11b26facf0daf99b6c. Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.094092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 474/934] 10.222.0.33 () {60 vars in 1140 bytes} [Wed Jun 15 00:22:43 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:44.100008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:44.115222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:44.195865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c209e-e086-4918-9c67-5e39c369d00c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:44.231796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:44.232588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.232863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.233128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.241712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:44.243147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-51276893-0433-4400-8de6-c96833a6e603 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:44.257777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:44.259985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:44.262179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 461/935] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:44.269494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:44.295786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:44.379514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28601-3313-4474-9ca4-10500b62cac5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:44.419614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:44.420424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.420853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.421221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.422081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:44.422482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:44.424252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:44.446647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d5d9-bb2e-4736-af5d-b69a595c3b27 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.447075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d5d9-bb2e-4736-af5d-b69a595c3b27 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.447472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d5d9-bb2e-4736-af5d-b69a595c3b27 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.448985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 462/937] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:44.449166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 475/937] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:44.454395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.454681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.454995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.479818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:44.481264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:44.563630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 463/938] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:44.586673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:44.672385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:44.693931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b506140-292f-4dd1-ae98-576f34ff6add None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:44.730563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:44.731428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.731704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.731973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.743999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:44.752289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:44.753397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:44.754536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 476/939] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:22:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:44.759746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:44.775088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:44.864326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bda7f-2aa6-4075-bb71-1b2b01e4c8b1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:44.899263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:44.900113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:44.900493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:44.901014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:44.901765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:44.902068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:44.903454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:44.912671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 464/940] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:22:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:44.917835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.023558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c1983-3dd5-42f6-b916-2e1530b9df64 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.064260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.064260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.064260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.064260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.074800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.078233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.078233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1e7f5091-9831-4fcb-9231-b350236698b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:45.096382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:45.097743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:45.098640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:45.098640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:45.098640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:45.098640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:45.098640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 477/941] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:45.104340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.201601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.219048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b242-4df1-4c00-8c23-2a30950490b5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.231952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1cf82-6a8d-4257-a869-a4a59b34a091 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.232234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1cf82-6a8d-4257-a869-a4a59b34a091 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.232504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1cf82-6a8d-4257-a869-a4a59b34a091 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.233868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 478/942] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:45.241181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.241181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.241181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.257302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:45.259606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:45.261010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.261667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.261946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.262217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.263124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:45.263430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.265582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.280705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 465/943] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:45.285821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.362816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 479/944] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:45.373898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.413465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e49aeb-0681-4010-99c1-3b8bcb27e0fb None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.457912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.459544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.459660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.460008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.463445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.468904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.470426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:45.494078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:45.495061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:45.496009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:45.497321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc65d0-9045-4209-9718-3ec94fffb9c6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.500806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 466/945] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:45.501121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.528171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.545526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.546550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.547070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.548044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.548884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.551363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:45.567205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:45.568739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:45.569576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:45.572771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 480/946] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:45.578740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.627029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f16ee-1a1c-4997-a9e8-36520c2c3dbb None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.670198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.671171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.672415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.672909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.675724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:45.676040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.677509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.704816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 467/947] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:45.706245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.715503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466ed5-3ae4-478e-8cbf-5bdafcc6ee89 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.753576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.754238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.754527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.754804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.756141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:45.756449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.757815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.767879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 481/948] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:45.770132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.781948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.837380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:45.838991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167f89f-9cd4-4d0a-92d5-6f4af215749d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.880488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.881656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.882168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.882659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.893970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.895739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.897251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c69ae-829a-4904-a749-66decc18b8e5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:45.920509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:45.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:45.922358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:45.922358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:45.922358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:45.922358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:45.928898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 468/949] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:45.929923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:45.933932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:45.935427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:45.935731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:45.936015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:45.936740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:45.937055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:45.938447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85131a-5993-4522-840b-bdd801ab66fa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:45.950820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 482/950] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:46.049243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91462-b246-4789-bf10-5abc93c9d823 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.058191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.088786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.089378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.089659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.089934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.090669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:46.091017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.092404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4354ea2-5c0f-47d0-8508-819d598bbafa None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.105102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88c72d-58f9-4ec9-b099-61a80aec2543 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.105624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88c72d-58f9-4ec9-b099-61a80aec2543 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.105624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88c72d-58f9-4ec9-b099-61a80aec2543 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.106523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 469/951] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:46.107118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 483/952] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:46.112677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.112958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.113239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.113542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88c72d-58f9-4ec9-b099-61a80aec2543 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.133673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:46.134907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:46.221801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 470/953] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:46.236786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88c72d-58f9-4ec9-b099-61a80aec2543 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.245288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.256078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.279395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.280380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.280831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.281333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.288713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.290290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:46.305696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:46.306809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:46.307787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 484/954] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:46.313928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.375481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.385667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb346d65-633d-4e74-aa2c-43b16ed163b9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.425370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198cd48-fb66-4c33-81c3-cb8e97e6a2d7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.431131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.432231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.432735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.433647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.434509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.436473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:46.452999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:46.454154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:46.455068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:46.456748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 471/955] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:46.463619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.468138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.468470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.468754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.469026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.471302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:46.471302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.472333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.489799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 485/956] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:46.497138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.533369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.596721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844c21-03e5-46e8-aa77-02e044152de8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.604777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd32194-72f9-4b3e-b84c-f303e9f5c458 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.641639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.642322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.643013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.643295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.643568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.644281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:46.644513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.644613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.645081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.645555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2eb34259fb6a4963b9277d2607c21468/roles/5adee2b2a8224db8aaca0778a60b9ffb` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.645937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.649007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=2eb34259fb6a4963b9277d2607c21468, role_id=5adee2b2a8224db8aaca0778a60b9ffb)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.651129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771962-b3be-47b1-8aae-a98911f96ef3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.656509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 472/957] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:46.658457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.667631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 486/958] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:22:46 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/2eb34259fb6a4963b9277d2607c21468/roles/5adee2b2a8224db8aaca0778a60b9ffb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:22:46.695964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.791247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107a3a-c894-4272-a798-20da68582ec0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:46.792778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfe54-a30e-485b-9fca-6aa98fe85b79 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.793259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfe54-a30e-485b-9fca-6aa98fe85b79 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.793853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfe54-a30e-485b-9fca-6aa98fe85b79 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.795736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 487/959] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:46.805134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.805509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.805867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.824366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:46.825889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:46.830986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:46.831975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:46.832498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:46.832949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:46.834441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:46.834950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:46.836650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:46.843069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 473/960] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:46.848711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.918973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 488/961] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:46.936484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:46.959944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.962386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:46.977790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0642d82-e809-4e01-9fc5-b3bdc38ce916 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:47.020841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:47.022077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:47.022360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:47.022636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:47.024475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:47.025863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:47.049846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d040f-5595-4973-98e1-621df8468a67 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:47.053410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 474/962] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:22:47.108955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:47.109677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:47.109925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:47.110195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:47.111131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:47.112670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:47.129034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.sql.core [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 00:22:47.129034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 00:22:47.129034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [parameters: {'user_id': 'f3181a3d83384185a2a2cbf0c64be5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 00:22:47.129034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Jun 15 00:22:47.137783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:22:47.138728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application yield resource Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise self.value Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application session.commit() Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 00:22:47.139715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.flush() Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application compat.raise_( Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 00:22:47.140629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 00:22:47.141555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 00:22:47.142431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f3181a3d83384185a2a2cbf0c64be5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:47.144935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 00:22:47.147182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:47.148330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 489/963] 10.222.0.33 () {60 vars in 1131 bytes} [Wed Jun 15 00:22:46 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:47.149888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:47.266852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c584fb-f459-40aa-b8a3-673eebf22ddd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:47.302341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:47.303187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:47.303470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:47.303741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:47.311386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:47.312822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-39870ff7-7606-4963-8e05-05068c92e673 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:47.327516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:47.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:47.329642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 475/964] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:22:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:47.337337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:47.425564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:47.447050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef924d78-aa94-4dd1-9121-873d3705342d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:47.482096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:47.483124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:47.483688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:47.484240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:47.485237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:47.485738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:47.487597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826beeeb-1fb1-488f-914d-ba5827abd215 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:47.532246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 490/965] 10.222.0.33 () {56 vars in 1101 bytes} [Wed Jun 15 00:22:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:48.174718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fd99-9550-4aab-82cf-0d94b38bf785 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:48.175259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fd99-9550-4aab-82cf-0d94b38bf785 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:48.175337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fd99-9550-4aab-82cf-0d94b38bf785 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:48.176657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 476/966] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:48.182340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc0c7-72f6-4b5f-b284-d00084ca6843 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:48.183019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc0c7-72f6-4b5f-b284-d00084ca6843 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:48.183019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc0c7-72f6-4b5f-b284-d00084ca6843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:48.200422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c62dc0c7-72f6-4b5f-b284-d00084ca6843 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:48.201934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c62dc0c7-72f6-4b5f-b284-d00084ca6843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:48.274923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 491/967] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:48.295902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fd99-9550-4aab-82cf-0d94b38bf785 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:48.300781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:48.402252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fd99-9550-4aab-82cf-0d94b38bf785 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:48.436874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:48.437450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:48.437725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:48.437991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:48.438717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:48.440120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d10d1f-0bfb-4855-a857-56b1f52b895e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:48.462805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 477/968] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:48 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:48.848795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bff65-0165-4581-8ff1-920ea41de140 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:48.849178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bff65-0165-4581-8ff1-920ea41de140 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:48.849589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bff65-0165-4581-8ff1-920ea41de140 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:48.851059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 492/969] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:48.856484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:48.856771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:48.857042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:48.874238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:48.875453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:48.935391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:48.958818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 478/970] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:48.976331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bff65-0165-4581-8ff1-920ea41de140 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.046651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.078611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bff65-0165-4581-8ff1-920ea41de140 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.112174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.112901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.113182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.113453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.124208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ddefdb15-e7b9-46c4-b4f5-22dd41ed79ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:49.133046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:49.133988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:49.135318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:49.137133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 493/971] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:22:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:49.140176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.243757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c5fdd-e4a9-4004-a237-bf9b9038f9f5 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.244353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c5fdd-e4a9-4004-a237-bf9b9038f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.244353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c5fdd-e4a9-4004-a237-bf9b9038f9f5 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.245844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 494/972] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:49.250659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.250980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.251255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.252773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9a72-aad0-4e9a-b8c9-516ed3ec7f3a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.267840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:49.269028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:49.291175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.291840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.292125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.292402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.293150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:49.293461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.294848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.308402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 479/973] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:49.309441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.341552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 495/974] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:49.363777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.364252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.415351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db106c0c-c99c-4f56-b773-98456aaae30d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.426296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.455799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.456615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.456896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.457167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.469519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.470992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.477177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9346cd-d377-4438-8d9c-4aa9c303aa4c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:49.489687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:49.490604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:49.491524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:49.491524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:49.491524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:49.491524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:49.491524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 480/975] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:49.497532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.522136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.523226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.523506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.524170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.525091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.527108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:49.543327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:49.544341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:49.545213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:49.550949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 496/976] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:49.553341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.604917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c05bd-c0c0-42c8-8cc6-0f15908c22b6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.608471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.644802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.645482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.645769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.646073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.646825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:49.647141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.648520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.665018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 481/977] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:49.666051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.669482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645abe9-930d-4c6a-95f4-bd02a0ec34c1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.697245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.712540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.713526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.713958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.714235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.715006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:49.715315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.716767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.722223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 497/978] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:49.728551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.781440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb529581-8ddc-4aaf-b799-db4f215e35d2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.822491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.823742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.824025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.824386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.837752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.839230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.852863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f16b6a-8f19-455a-9480-89217af7d3b9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:49.855717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:49.856657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 00:22:49.857626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:49.862083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 482/979] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:49.862736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.893081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.897385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:49.898004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:49.898449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:49.898728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:49.899465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:49.899771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:49.901156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:49.906434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 498/980] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:49.912437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:49.971638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:49.986149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc68ac-c02f-496e-9ab3-da5feef82542 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.026083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf4b5a-3800-4778-9eaf-98ef2342e582 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.035175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.035846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.036124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.036394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.039491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:50.039491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.039491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.062147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 483/981] 10.222.0.33 () {56 vars in 1149 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:50.069269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.071082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.071745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.072027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.072286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.073100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] RBAC: Adding query filter params (interface=admin, service_id=c35ac621c87343a284b20d3ae747802f, region_id=RegionOne) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:50.073413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.074806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.082753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 499/982] 10.222.0.33 () {56 vars in 1249 bytes} [Wed Jun 15 00:22:49 2022] GET /identity/v3/endpoints?service_id=c35ac621c87343a284b20d3ae747802f&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:50.088649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.166470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.191838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f06fa-2a3c-47bc-aa0e-184f9f1ddf18 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.216349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2103aa-a9be-4f3c-959c-c3977398256a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.233751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.241405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.242304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.242658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.243027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.259314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.260321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.260600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.260873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.261767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.263604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.270417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 500/983] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:22:50.321247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.322750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fc78a-c549-4224-8126-e133dd42c26e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.412705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.434754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 484/984] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:22:50.440425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.555052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c924c-b151-4b76-ba92-dbb6a909692e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.596435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.597233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.597519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.597794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.609045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.610494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.616580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97163f5-6949-4270-89e4-db9220844c76 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.616866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97163f5-6949-4270-89e4-db9220844c76 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.617138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97163f5-6949-4270-89e4-db9220844c76 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.618624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 485/985] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:50.623439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.623729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.624003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:50.633926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:50.634861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:50.635773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:50.635773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:50.635773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:50.635773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:50.635773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 501/986] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:50.640714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.642436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:50.643655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:50.676525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.723471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 486/987] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:50.746414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.747370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9893-5ab8-4e17-9042-dd810e5cdcd9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.786028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.787244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.787571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.787800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.788923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:50.788923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.790909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.809183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 502/988] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:50.814604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.866868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954005-48d2-40a6-b2fc-5fc02e1ec867 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.881599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.903365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.904161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.904437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.904712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.905363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.906729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.908658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:50.919191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf01c6-7d7f-450a-9350-98593819fbe8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:50.919816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:50.920779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:50.921629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 487/989] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:50.926681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:50.958204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:50.959083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:50.959365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:50.959643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:50.966326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:50.967797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:50.982640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:50.983595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:50.984491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 503/990] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:50.989654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:51.031841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215760-592c-485a-872f-7bbf3891a82b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.068328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.068993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.069508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.069621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.070237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:51.070534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.071939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.077108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 488/991] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:51.082371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:51.094312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f3d83-5a9f-4b51-9f96-85bb9b773a01 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.130511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.131260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.131538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.131809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.132590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:51.132892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.134368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.134878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:51.149016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 504/992] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:51.154910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:51.170589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:51.200746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7fad2-2fb6-45c3-bba0-58b332345519 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.236633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.237310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.237592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.237864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.238557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] RBAC: Adding query filter params (type=compute) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:51.238887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.240436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.246219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 489/993] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:51.251618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:51.261394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eb60-1b79-466a-9ff4-a226dcb817c4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.297774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.298430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.298710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.299013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.299848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] RBAC: Adding query filter params (role.id=5adee2b2a8224db8aaca0778a60b9ffb, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=2eb34259fb6a4963b9277d2607c21468) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:51.300160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.301626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.308416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 505/994] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:51 2022] GET /identity/v3/role_assignments?user.id=2eb34259fb6a4963b9277d2607c21468&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=5adee2b2a8224db8aaca0778a60b9ffb => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:51.359515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5a083-6d1c-44a2-a252-41db51d019fe None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.362217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:51.400468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.401143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.401419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.401746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.402550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] RBAC: Adding query filter params (interface=public, service_id=be9949efc1e64196bbefc9fbe2e2252f, region_id=RegionOne) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:51.402885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.404261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b212f-bc4e-459f-823f-56525fe062c9 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.416301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 490/995] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:51 2022] GET /identity/v3/endpoints?service_id=be9949efc1e64196bbefc9fbe2e2252f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:51.418170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:51.504437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:51.529126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5709a6a-f506-4974-be76-248a12476951 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:51.562841 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:51.563511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:51.563792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:51.564066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:51.564758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:51.566120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168f089-297c-4611-b248-ec080fb48c8e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:51.573059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 506/996] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:51 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:22:52.434083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040388-a9fc-4eb7-a388-8e51b2b2a94d None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:52.434646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040388-a9fc-4eb7-a388-8e51b2b2a94d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:52.434646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040388-a9fc-4eb7-a388-8e51b2b2a94d None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:52.436173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 491/997] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:52.441098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:52.441477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:52.441658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:52.458359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:52.459542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:52.537124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 507/998] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:52.559880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab040388-a9fc-4eb7-a388-8e51b2b2a94d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:52.664757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab040388-a9fc-4eb7-a388-8e51b2b2a94d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:52.704526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:52.705262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:52.705542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:52.705814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:52.706469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:52.707871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:52.720432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:52.721409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:52.722252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 492/999] 10.222.0.33 () {56 vars in 1126 bytes} [Wed Jun 15 00:22:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:52.728343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:52.799045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:52.830381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a552d5e-5f0b-49bc-9a4c-5b15ff4a2f89 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:52.865722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:52.866382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:52.866663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:52.866965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:52.867672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:52.867976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:52.869338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:52.878655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 508/1000] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:52.879697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:52.912247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:52.981965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0ca8d-2107-4db7-b530-e9993dd37d72 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:53.015550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a13285-547f-4790-8716-332535cfa613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:53.016213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13285-547f-4790-8716-332535cfa613 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.016494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13285-547f-4790-8716-332535cfa613 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.016767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13285-547f-4790-8716-332535cfa613 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.017466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13285-547f-4790-8716-332535cfa613 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:53.017772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13285-547f-4790-8716-332535cfa613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:53.019166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13285-547f-4790-8716-332535cfa613 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:53.029132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 493/1001] 10.222.0.33 () {56 vars in 1122 bytes} [Wed Jun 15 00:22:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:53.030277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:53.132583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adbecb-373d-4814-b9ac-e12aff75ded5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:53.166588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:53.167271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.167557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.167828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.168572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:53.169934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcb8a6-4944-45fa-b333-0da1b045f067 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:53.173741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:53.198809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 509/1002] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:53.456149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabb0-59fe-452d-99ce-22e704648e85 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.456646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabb0-59fe-452d-99ce-22e704648e85 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.456646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebabb0-59fe-452d-99ce-22e704648e85 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.458208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 494/1003] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:53.463241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f8c-3157-41d2-860a-a90c22c7bfc1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.463627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f8c-3157-41d2-860a-a90c22c7bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.463803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f8c-3157-41d2-860a-a90c22c7bfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.483144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7f8d5f8c-3157-41d2-860a-a90c22c7bfc1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:53.484320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7f8d5f8c-3157-41d2-860a-a90c22c7bfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:53.559721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 510/1004] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:53.581726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebabb0-59fe-452d-99ce-22e704648e85 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:53.681629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:53.689540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebabb0-59fe-452d-99ce-22e704648e85 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:53.707269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e439a-16d5-49b8-9d55-a4bef75e549d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.707530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e439a-16d5-49b8-9d55-a4bef75e549d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.707746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e439a-16d5-49b8-9d55-a4bef75e549d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.709523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 511/1005] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:53.723016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.723571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.723779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.732011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:53.732806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.733090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.733366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] PATH_INFO: `/v3/services/image` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.734022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:53.735424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:53.743869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:53.745237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:53.747720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:53.748676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:53.749553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 495/1006] 10.222.0.33 () {56 vars in 1105 bytes} [Wed Jun 15 00:22:53 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:53.754845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:53.821990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 512/1007] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:53.844229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:53.862405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795326b7-1555-47d5-b1ce-bb2dace56144 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:53.896390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:53.900295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:53.901019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.901307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.901584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.902308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] RBAC: Adding query filter params (name=image) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:53.902673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:53.904030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:53.914703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 496/1008] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:53.915959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:53.942766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:53.951322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730919b-6e50-43e5-9777-4884a42e8771 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:53.987742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:53.988523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:53.988799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:53.989059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:53.990327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:53.991721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:54.003668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:54.004728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:54.005907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 513/1009] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:54.010595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:54.028883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e7840-ca53-409f-99c8-06a69e80918e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:54.065086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:54.065746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:54.066029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:54.066411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:54.067039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:54.067349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:54.068796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:54.082824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 497/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:54.133359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340af20-6078-4a7f-bbee-b7a9cdbb9944 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:54.149777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:54.172569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:54.173241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:54.173524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:54.173795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:54.174522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:54.174856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:54.176244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:54.187229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 514/1011] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:54.188313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:54.282322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:54.297093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e25067-860e-4b55-b615-c822cf69e6d6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:54.333077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:54.334006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:54.334444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:54.334715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:54.335455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:54.335760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:54.337150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418a7ca-3889-4486-a3b4-8ac5f05d66b8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:54.343128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 498/1012] 10.222.0.33 () {56 vars in 1112 bytes} [Wed Jun 15 00:22:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:54.348804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:54.452050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f71a-1431-4fb2-9b29-57f745141a50 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:54.494120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:54.494120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:54.494859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:54.494859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:54.495461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:54.496838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04b611-a1f9-4755-9f26-2babfac124dc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:54.522805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 515/1013] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:22:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:55.354417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a8b4-6fe7-416e-8407-0f9dd9c33993 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:55.354972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a8b4-6fe7-416e-8407-0f9dd9c33993 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:55.355049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a8b4-6fe7-416e-8407-0f9dd9c33993 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:55.356528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 499/1014] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:55.361463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:55.361763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:55.362045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:55.378719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:55.379939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:55.407597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:55.456609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 516/1015] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:55 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:55.484060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a8b4-6fe7-416e-8407-0f9dd9c33993 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:55.587662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a8b4-6fe7-416e-8407-0f9dd9c33993 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:55.621560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:55.622282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:55.622563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:55.622861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:55.624107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:55.625598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23050940-a553-4229-a955-f7eafab38f07 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:55.630944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 500/1016] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:55.633126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:55.737910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e98ac3-b0fb-4d04-a0d2-09a90d9f6013 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:55.772953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:55.772953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:55.773262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:55.773346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:55.774031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:55.775558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927cdf1-7dea-4f18-9508-90274247c7eb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:55.806873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 517/1017] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:55 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:22:56.253233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54868c-6dc6-44ae-9df3-4f2eda4fb3a7 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:56.253603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54868c-6dc6-44ae-9df3-4f2eda4fb3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:56.253785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54868c-6dc6-44ae-9df3-4f2eda4fb3a7 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:56.255849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 501/1018] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:56.260827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf11c96-41fb-4a80-a06d-4226a65ba2d2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:56.261115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf11c96-41fb-4a80-a06d-4226a65ba2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:56.261387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf11c96-41fb-4a80-a06d-4226a65ba2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:56.278387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4bf11c96-41fb-4a80-a06d-4226a65ba2d2 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:56.279593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4bf11c96-41fb-4a80-a06d-4226a65ba2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:56.287217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:56.360115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 518/1019] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:56 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:56.383558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54868c-6dc6-44ae-9df3-4f2eda4fb3a7 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:56.405690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:56.488142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54868c-6dc6-44ae-9df3-4f2eda4fb3a7 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:56.527288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:56.527864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:56.528148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:56.528426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:56.529155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:56.530532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528a962-9f3f-40f7-b70f-902148584ffd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:56.547001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 502/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:22:56 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:22:56.731241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bf45-46d3-4b32-8a15-f8cc7396a557 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:56.731754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bf45-46d3-4b32-8a15-f8cc7396a557 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:56.731844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bf45-46d3-4b32-8a15-f8cc7396a557 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:56.733156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 519/1021] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:22:56.738589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:56.738963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:56.739245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:56.757088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:56.758293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:56.830025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 503/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:56.855344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bf45-46d3-4b32-8a15-f8cc7396a557 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:56.967182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bf45-46d3-4b32-8a15-f8cc7396a557 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.003548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.004367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.004647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.004921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.015235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:22:57.023482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:57.024452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:57.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 520/1023] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:22:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:57.032401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.041941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:57.139577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d86d68-ddc3-4789-babf-b6a02194bee5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.175060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.175723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.176001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.176273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.177039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:57.177369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:57.178730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:57.191501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 504/1024] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:57.192453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.195498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:57.305671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10c5e-806c-46a5-b0fe-63d3e906b99b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.340855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.341652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.341933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.342204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.353764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:57.355265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:57.372365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:57.373278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:57.374189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:57.374189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:57.374189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:57.374189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:57.374189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 521/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:57.380600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.450372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:57.491928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b5865-f9f5-4dec-b434-697bc7d94832 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.525977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.526629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.526932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.527209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.527920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:57.528225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:57.529578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:57.542918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 505/1026] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:57.547278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.606960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:57.655483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b392fa-38b5-4304-867e-7d2072ab4e7b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.689869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.690675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.690984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.691257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.699868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:57.701359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:22:57.715699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:57.716691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:57.717689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 522/1027] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:57.724428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.827625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d1acb-3d07-41d6-9d43-4cae331bf207 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:57.854716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:57.864857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:57.865526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.865808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.866080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.866940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:57.867251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:57.868664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48619d82-7cfa-40cf-8641-f0523b1f9253 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:57.890673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 506/1028] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:57.891814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:57.975367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da144f37-0f72-461a-8ace-753080f9499a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.975656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da144f37-0f72-461a-8ace-753080f9499a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.975932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da144f37-0f72-461a-8ace-753080f9499a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.977557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 507/1029] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:57.982488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:57.982856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:57.983078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:57.999677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:58.000852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:58.001536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376e41-c69d-4322-8bb3-24f996d63f11 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.016599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.037978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.038617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.038921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.039193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.074066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 508/1030] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:22:58.098602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.112319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.113768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.203872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f9fb-c6b9-407f-9829-7d3d255e766a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.215672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 523/1031] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:22:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:22:58.221156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.227352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.236612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.247100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.247899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.248180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.248450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.249743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.251254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.252980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 509/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:58.258290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.329700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00965-c304-403c-8d24-1fdf36e990df None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.365230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.366016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.366291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.366559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.367563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd424-b915-412b-96df-8b6b6dd79181 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.376503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.377942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:22:58.395145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:58.396282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:58.397212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:22:58.397212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:22:58.397212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:22:58.397212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:58.397212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 524/1033] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:58.401813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.408140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.408798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.409072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.409337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.410044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.412059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.433276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 510/1034] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:22:58 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:22:58.501325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.540990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d3637-b85b-40b4-bef6-8f5c33c76e6c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.578026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.578705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.579012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.579289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.580025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:58.580334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.581733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff57e6-fc9d-4948-880f-31fbc7c259b2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.594654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 525/1035] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:58.600621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.650017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.720096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3656b8-b02c-4859-b493-f4e4a32922a5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.751761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268109dc-87d1-44bf-91c4-4294a9201ffa None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.751972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268109dc-87d1-44bf-91c4-4294a9201ffa None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.752257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268109dc-87d1-44bf-91c4-4294a9201ffa None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.753640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 526/1036] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:22:58.756526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.757528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.758030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.758478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.758605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.758805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.759102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.766610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.768316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.776407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:22:58.777589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:22:58.783984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:58.784898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:22:58.785997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:22:58.792870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 511/1037] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:22:58.794468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.848468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 527/1038] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:22:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:22:58.871423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.902791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.906974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b754df-9412-4088-9adf-543df081dd5a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:58.928025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:58.948436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:58.949101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:58.949386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:58.949872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:58.950443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:58.950752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:58.952284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:58.965150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 512/1039] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:58.970817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:58.975341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e05f0-3a48-456f-ac41-22d0c52f6338 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.014740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.015885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.016168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.016439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.017406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.019231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:22:59.034966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:22:59.035972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:22:59.042480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 528/1040] 10.222.0.33 () {56 vars in 1105 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:22:59.043714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:59.074154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892888a-6249-4848-b25c-86aacd01630b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.111033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.111963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.112245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.112509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.113337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] RBAC: Adding query filter params (role.id=5adee2b2a8224db8aaca0778a60b9ffb, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:59.113647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.115199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.130811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 513/1041] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:22:58 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=5adee2b2a8224db8aaca0778a60b9ffb => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:59.167724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed694-dda7-4928-a741-e0898b394da3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.189894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:59.206839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.207520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.207799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.208639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.209369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] RBAC: Adding query filter params (name=image) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:59.209678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.211142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.216749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 529/1042] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:59.222528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:59.251854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:59.332187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972c75-6dd5-4475-88ef-592a69051c33 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.367065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.367719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.367997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.368266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.368959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:59.369259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.370706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.381053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 514/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:22:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:22:59.382251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:59.486008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79cebf-73c1-4eb2-97da-9f1de103c053 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.519988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.521151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.521427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.521687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.522463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] RBAC: Adding query filter params (interface=public, service_id=19053ef3ff444b4981e6e50323cfccea, region_id=RegionOne) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:22:59.522767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.524155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4412-e74a-4a81-a88f-feeb3af3bf3d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.534000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 530/1044] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:22:59 2022] GET /identity/v3/endpoints?service_id=19053ef3ff444b4981e6e50323cfccea&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:22:59.535145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:22:59.627808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:22:59.641605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214a45-194e-4ba3-afd9-fc6eda649bd0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:22:59.676327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:22:59.677019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:22:59.677299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:22:59.677570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:22:59.678251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:22:59.679685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd3044-03b1-4a71-8675-7776f8d14609 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:22:59.694816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 515/1045] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:22:59 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:23:00.639947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80079f-f7b5-4ee0-b5a1-ed4d05ea5b9b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:00.640511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80079f-f7b5-4ee0-b5a1-ed4d05ea5b9b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:00.640511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80079f-f7b5-4ee0-b5a1-ed4d05ea5b9b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:00.641837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 531/1046] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:00.647153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:00.647580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:00.647755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:00.668117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:00.669545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:00.740625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 516/1047] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:00.764489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80079f-f7b5-4ee0-b5a1-ed4d05ea5b9b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:00.817178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:00.871543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80079f-f7b5-4ee0-b5a1-ed4d05ea5b9b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:00.905609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:00.906399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:00.906683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:00.906985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:00.918179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:00.925855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:00.926767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:00.927654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:00.931125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 532/1048] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:00.933738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.016382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.035894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747682a-951c-44f8-ad97-408b43c9d3d8 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.069526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.070183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.070458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.070725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.071491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:01.071796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.073151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d76-e9cc-4a79-95ca-b0859147d65b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.086574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 517/1049] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:01.087634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.187864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a4c4-a9f5-4d3b-b7fd-861010e1a165 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.198756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.226811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.227677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.227952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.228216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] PATH_INFO: `/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.241586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] RBAC: Authorizing `identity:get_user(user_id=c98b38af0bb842d9ae9d1ef3a69ae42b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.243221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211de6-3262-42d7-8f90-503087eedc1a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.243633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a89db-7f17-4bbb-b30a-abe45972a6f6 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.243926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a89db-7f17-4bbb-b30a-abe45972a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.244209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a89db-7f17-4bbb-b30a-abe45972a6f6 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.245041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 533/1050] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:01.245606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 518/1051] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:23:01.251341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.251623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.251898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.255821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a89db-7f17-4bbb-b30a-abe45972a6f6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.269031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:01.270139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:01.336761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 534/1052] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:01.359319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.363369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a89db-7f17-4bbb-b30a-abe45972a6f6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.400039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.401059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.401486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.401902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.417097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.454501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.462137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8d687-f337-4010-bab3-ca92cad7554c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.480871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.482323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.501266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.502081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.502365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.502623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.514554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:01.523204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:01.524139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:01.525066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 535/1053] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:01.530344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.597897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 519/1054] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:23:01.598990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.645687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1f5b-62fb-4ce6-8a21-3d27c5fc91fb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.648528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.684495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.685442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.685725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.686001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.686946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:01.687634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.689016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.698373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 536/1055] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:01.703642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.708566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11142702-9ced-4298-8a3e-4615b7abb4e8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.722759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.754255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.755741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.756024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.756299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.757597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.759112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.761099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 520/1056] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:01.768336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.822087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f134-7408-46aa-9749-4eb8cae6e688 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.855711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.856494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.856776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.857045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.868235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.869580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e004be0-ad8a-4f3d-8316-b0ecf06690f2 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:01.869689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.883176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6539d698-bce2-47ec-ae14-fd8a1c9d30fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:01.887290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:01.888195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:01.889137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:23:01.889137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:01.889137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:01.889137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:01.889137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 537/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:01.889672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee55a4-ac50-41a6-aec0-4a9567e1d90b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.889949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee55a4-ac50-41a6-aec0-4a9567e1d90b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.890214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee55a4-ac50-41a6-aec0-4a9567e1d90b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.891497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 538/1058] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:01.894187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee55a4-ac50-41a6-aec0-4a9567e1d90b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:01.909812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:01.910462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.910741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.911071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.911896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=6e038ce8a0d447d193a94e48b6eee195, user.id=c98b38af0bb842d9ae9d1ef3a69ae42b) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:01.912203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:01.913625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b87afa-be8c-46ed-bfe2-1b34ec051c6e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:01.930806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 521/1059] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/role_assignments?user.id=c98b38af0bb842d9ae9d1ef3a69ae42b&scope.project.id=6e038ce8a0d447d193a94e48b6eee195&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:01.930806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:01.930806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:01.930806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:01.938284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:01.939552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:01.979577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.021001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee55a4-ac50-41a6-aec0-4a9567e1d90b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.027351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 522/1060] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:02.051066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.063687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.064739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.065253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.065736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.066684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:02.067094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.068541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.081812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 539/1061] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:02.087088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.167592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d818a69-e3b6-43cc-b8b0-042e926f56e6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.175853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.195230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a9916-a385-45ca-bfc2-247bc71427f2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.213803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.214891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.215383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.216416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] PATH_INFO: `/v3/services/image` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.217540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.219844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.225163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.235145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:02.235631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:02.236595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 00:23:02.237434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:02.238194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.238194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.238194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.238375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 523/1062] 10.222.0.33 () {56 vars in 1105 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:02.244281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.249178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.250656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:23:02.265587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:02.266553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:02.267543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 540/1063] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:02.272461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.351898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef0af3-9216-4486-9e13-60de07d5d794 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.381586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0220de-9de8-4c88-a11e-bd19f6d03a17 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.387404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.388074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.388356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.388629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.389362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] RBAC: Adding query filter params (name=image) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:02.389669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.391506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.397945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 524/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:02.403563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.420187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.420860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.421140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.421414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.422250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:02.422573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.424030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.440953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 541/1065] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:02.446122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.446805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.502103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.520203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953097d5-7b0f-48b2-b6d5-301b15f41b79 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.557324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.558157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.558434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.558717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.559431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] RBAC: Adding query filter params (type=image) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:02.559734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.560843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e33d6-e039-46f4-937f-e5d25edf15ae None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.561091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.567054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 525/1066] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:02.572291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.597447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.598236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.599130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.599401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.611433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.612809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:02.634494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:02.636400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:02.638166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:23:02.638166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:02.638166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:02.638166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:02.638723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 542/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:02.641458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.680985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.691441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82619ac-3509-4f09-bda5-97517b19454d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.729208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.729853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.730129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.730398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.731165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.732506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.745583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:02.750847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056801-0aba-43f2-8596-99183f268521 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.752158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 526/1068] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:23:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:23:02.808517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.809516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.809972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.810417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.811416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:02.811931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.813532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:02.827437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 543/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:02.833976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:02.940856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4c0e-55b5-4e82-bd61-7380dd84a6a5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:02.975989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:02.977045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:02.977318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:02.977587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:02.984522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:02.985945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:23:03.000464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:03.001561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:03.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 527/1070] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:03.009911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:03.110270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716fb19-5b5a-4509-8800-b915e5a5b39e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:03.144438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:03.145101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:03.145406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:03.145678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:03.146457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:03.146764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:03.148197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40d375-8ad7-4799-bb2c-4013d31310ae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:03.160706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 544/1071] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:03.166367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:03.182467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:03.269400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4fdf-31b2-4276-bd1d-a22416e67be6 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:03.302930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:03.303839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:03.304123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:03.304390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/5adee2b2a8224db8aaca0778a60b9ffb` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:03.306541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=42f57540b6cd47559628031b441a1eb9, role_id=5adee2b2a8224db8aaca0778a60b9ffb)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:03.308118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570ac2-7418-43d5-9ad2-26cd9c84e041 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:03.324997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 528/1072] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:23:03 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/42f57540b6cd47559628031b441a1eb9/roles/5adee2b2a8224db8aaca0778a60b9ffb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:23:04.048495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ded3c-d7f6-4f20-a621-5036cecc1556 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.049017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ded3c-d7f6-4f20-a621-5036cecc1556 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.049017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ded3c-d7f6-4f20-a621-5036cecc1556 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.050578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 545/1073] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:04.055675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.056058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.056225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.072715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:04.073893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:04.144078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 529/1074] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:04.166947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ded3c-d7f6-4f20-a621-5036cecc1556 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:04.185143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:04.269263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ded3c-d7f6-4f20-a621-5036cecc1556 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:04.306404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:04.307140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.307427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.307698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.318628 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:04.326251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:04.327289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:04.328194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:04.331133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 546/1075] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:04.334085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:04.337256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:04.438497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39dba0-1a70-4804-8f6d-61d81dbf4612 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:04.473382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:04.477248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.477248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.477248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.477248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:04.477248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:04.477634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:04.491935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 530/1076] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:04.493754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:04.574256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:04.598701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0bcee-00f9-4eb0-8ce7-716502f85563 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:04.638464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:04.639972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.640338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.640615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] PATH_INFO: `/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.641846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] RBAC: Authorizing `identity:get_user(user_id=c98b38af0bb842d9ae9d1ef3a69ae42b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:04.643399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:04.649692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 547/1077] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:04 2022] GET /identity/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:04.651346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:04.718287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:04.756008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e346e-c19c-4227-acfd-79db0aa0df95 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:04.788854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:04.789702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.789980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.790250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.791550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:04.792971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e90af-275c-4b6c-ac5b-e3197a01f403 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:04.798898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 531/1078] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:04 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:04.800896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:04.908501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd58287-7a49-4560-bc88-acf3655caeca None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:04.921087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413493ea-730d-4ba0-90a4-53d231f79914 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.921367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413493ea-730d-4ba0-90a4-53d231f79914 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.921744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413493ea-730d-4ba0-90a4-53d231f79914 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.926958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 532/1079] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:23:04.931338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.933833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.934722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.951994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:04.952907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:04.953186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:04.953458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195/users/c98b38af0bb842d9ae9d1ef3a69ae42b/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:04.955225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:04.955453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e038ce8a0d447d193a94e48b6eee195, user_id=c98b38af0bb842d9ae9d1ef3a69ae42b, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:04.956782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:04.956892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:04.971718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 548/1080] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:23:04 2022] PUT /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195/users/c98b38af0bb842d9ae9d1ef3a69ae42b/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:23:05.037054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 533/1081] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:05.060325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:05.064351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.173999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4958dac-8031-48ca-938a-f6e3e9bfdb61 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:05.208521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:05.209167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.209452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.209728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.280055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:05.281507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b384ecf-9892-48e0-b385-eb9b0c775ffa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:05.284196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.385807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 549/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:05.391523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:05.427778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8075f-a2be-4b6a-b6e9-5308f1f271b3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.427990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8075f-a2be-4b6a-b6e9-5308f1f271b3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.428252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8075f-a2be-4b6a-b6e9-5308f1f271b3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.429783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 550/1083] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:05.434409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.434810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.435038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.452737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:05.453939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:05.457198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.504048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8463c-a8fb-4646-ad75-fad65bedc9ad None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:05.531192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 551/1084] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:05.543187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:05.543979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.544263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.544539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.555424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:05.555682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:05.556873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:05.574465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:05.575606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:05.576516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:23:05.576516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:05.576516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:05.576516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:05.580482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 534/1085] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:05.581262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:05.590640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.664891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9429-71eb-49e9-9584-aa28d3eb2213 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:05.686248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94c7c-0d68-4a99-a70c-4d06c256e193 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:05.701558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:05.702370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.702653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.702922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.703046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.716241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:05.725447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:05.726695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:05.727567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 552/1086] 10.222.0.33 () {56 vars in 1120 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:05.731526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:05.732195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.732475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.732749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.732843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:05.733471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:05.733778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:05.735216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:05.753941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 535/1087] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:05.842962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:05.852818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a9a95-f543-4247-aea2-bc6f60a75a80 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:05.887744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:05.888429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:05.888707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:05.888978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:05.889736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:05.890047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:05.891464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:05.905469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 553/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:05.907046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.011913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecad78-63bd-41bd-b43b-48000ac82a89 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:06.049005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:06.049794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:06.050079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:06.050353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:06.062270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:06.063800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:06.082280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:06.083820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:06.085190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:23:06.085190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:06.085190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:06.085190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:06.085190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 536/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:06.089725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.186999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a684-fd9b-4460-89b0-132a4d697c0f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:06.219126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:06.219772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:06.220049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:06.220317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:06.221023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:06.221324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:06.222667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:06.241564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 554/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:06.248031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.279916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:06.349903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe615db-59fc-4032-87a2-25fd5e2f81a7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:06.383100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:06.383882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:06.384164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:06.384435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:06.391634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:06.393061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:23:06.406682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:06.407835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:06.413638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 537/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Wed Jun 15 00:23:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:06.414381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.431737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:06.515138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1768d-3994-41d7-aa20-94a528f923e7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:06.548477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:06.549076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:06.549373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:06.549633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:06.550463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:06.550797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:06.552808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:06.569700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 555/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Wed Jun 15 00:23:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:06.575656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.670913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:06.683563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0c48-5b6e-4b84-a5cc-8c2097feb321 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:06.715742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:06.716374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:06.716655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:06.716928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:06.782799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:06.784255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:06.869042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:06.881957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 538/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:23:06.887745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:06.960906 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:06.986105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513827eb-7428-41fa-9292-34717045d4aa None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.018636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.019473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.019756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.020553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.031985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.033500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c8c839b9-4ebf-4fb8-9673-6e10b32e36ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:07.053088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:07.054206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:07.055140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:23:07.055140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:07.055140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:23:07.055140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:07.055140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 556/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:23:07.060149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.095544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1681-4a25-4b14-88a8-388883269e75 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.096051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1681-4a25-4b14-88a8-388883269e75 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.096144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1681-4a25-4b14-88a8-388883269e75 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.097456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 557/1095] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:23:07.103204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.103490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.103768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.121089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:07.122200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:07.161412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d55ed-7afa-4fb6-9043-145bec7a94c8 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.188393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 558/1096] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:23:07.194441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.195111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.195392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.195663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.196376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:07.196678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.198030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.209425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 539/1097] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:07.211958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.214621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.263518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.268538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.311514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b040254-6a0d-4e34-a47e-6fddcb558216 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.314487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0aad-0ea7-4dc0-8d01-e06c69daead5 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.345331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.346118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.346400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.346671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.347807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.348611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.348889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.349167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.355979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.356963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:07.357416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:07.364709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:07.365801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:07.367239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 559/1098] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:07.371749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:23:07.372379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:07.373246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:07.374162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 540/1099] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:07.378992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.470896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce01e3-71e9-46da-afe4-468f38927eb0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.479734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3ee39-a9b3-4e34-97c0-8a497502f10b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.486985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.490348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.507385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.508056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.508339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.508614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.509358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:07.509666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.511072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.516125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.516773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.517052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.517326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.518110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:07.518416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.519242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 560/1100] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:07.524211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.525555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.535411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 541/1101] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:07.540356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.624003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be32ba6-a761-4f2d-b0d8-2d2f75c785e2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.636975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95326cb2-e17a-470c-8e09-4e19615df96c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.658294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.659183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.659462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.659725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] PATH_INFO: `/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.660905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=c98b38af0bb842d9ae9d1ef3a69ae42b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.662312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.663957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 561/1102] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/users/c98b38af0bb842d9ae9d1ef3a69ae42b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:07.669513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:07.671180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.671819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.672106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.672380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.673207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] RBAC: Adding query filter params (role.id=5adee2b2a8224db8aaca0778a60b9ffb, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=42f57540b6cd47559628031b441a1eb9) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:07.673519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.674972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.681285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 542/1103] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/role_assignments?user.id=42f57540b6cd47559628031b441a1eb9&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=5adee2b2a8224db8aaca0778a60b9ffb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:07.719666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.777301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e7a06-7aa5-4754-a0e3-6dcfc4b27ab1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:07.812290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:07.812948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:07.813230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:07.813502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:07.885752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:07.887157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:07.932182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:07.989888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 562/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:23:07.995874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:08.029050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:08.099126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66993-ad1a-4b0e-833e-39ba7a93fdcc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:08.134766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:08.135643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:08.135968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:08.136188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:08.137361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:08.138799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694dc0-2dcc-4ff6-ae13-3ffab5ac97de None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:08.140906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 543/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:07 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:08.146588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:08.248297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2b53-2dec-40f9-870e-3d8dcb2ec4d8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:08.282078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:08.282726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:08.283041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:08.283316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:08.284142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=6e038ce8a0d447d193a94e48b6eee195, user.id=c98b38af0bb842d9ae9d1ef3a69ae42b) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:08.284451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:08.286420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f77d6-dc02-4ef3-b14b-f27bd7522cb9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:08.292386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 563/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:08 2022] GET /identity/v3/role_assignments?user.id=c98b38af0bb842d9ae9d1ef3a69ae42b&scope.project.id=6e038ce8a0d447d193a94e48b6eee195&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:10.409873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d592-f6f1-4c9e-8aa3-1c496004196a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:10.410513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d592-f6f1-4c9e-8aa3-1c496004196a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:10.410513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d592-f6f1-4c9e-8aa3-1c496004196a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:10.411816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 544/1107] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:10.416877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:10.417287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:10.417419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:10.433615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:10.434796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:10.437481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:10.499898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 564/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:23:10.524750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18d592-f6f1-4c9e-8aa3-1c496004196a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:10.620311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:10.624216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18d592-f6f1-4c9e-8aa3-1c496004196a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:10.657036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:10.657807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:10.658090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:10.658366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:10.659558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:10.661028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76f34f-e928-4a2a-9231-b911ea6bb0e2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:10.662576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 545/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:10.668153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:10.765450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4705-7ef5-4258-9705-822a67ffba77 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:10.798082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:10.798729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:10.799036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:10.799307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:10.800016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:10.801447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaea06-becd-46bc-957e-f31c85165a90 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:10.810633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:10.821337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 565/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:23:10 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:23:13.002995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc247e-f396-451a-9f0d-8114c0b11d3a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.003525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc247e-f396-451a-9f0d-8114c0b11d3a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.003525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc247e-f396-451a-9f0d-8114c0b11d3a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.005539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 546/1111] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:13.010638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.011103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.011209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.027442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:13.028616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:13.092351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 566/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:13.115278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc247e-f396-451a-9f0d-8114c0b11d3a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:13.211890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc247e-f396-451a-9f0d-8114c0b11d3a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:13.245127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:13.245967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.246247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.246521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] PATH_INFO: `/v3/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.247998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] RBAC: Authorizing `identity:get_role(role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:13.249378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:13.250702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 547/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:13 2022] GET /identity/v3/roles/87cd2d147f344fa99848788e1a735d8f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:13.256202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:13.351358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb15ca-85e7-424b-93e2-b6bb51e67e85 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:13.375953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:13.384627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:13.385478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.385759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.386032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] PATH_INFO: `/v3/users/530d570a2a714da5b2ebc0c36f68bee6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.398843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] RBAC: Authorizing `identity:get_user(user_id=530d570a2a714da5b2ebc0c36f68bee6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:13.400285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:13.401900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 567/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:13 2022] GET /identity/v3/users/530d570a2a714da5b2ebc0c36f68bee6 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:13.407532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:13.441485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:13.505374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076c7-e604-45e6-a4ca-ff194ec9c297 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:13.539577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:13.540151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.540431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.540704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.605492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:13.606955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:13.638117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:13.700080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 548/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:13.706080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:13.803595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700222d6-54ce-45f5-9d3d-3be75a347d14 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:13.836210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:13.837045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.837320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.837596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.838745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:13.840197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337f989-827c-4365-bd19-274c5a782af4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:13.841846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 568/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:13 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:13.848611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:13.943477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f25a-41fc-4b6f-bf91-92e8c13d8e06 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:13.975773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:13.976655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:13.976931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:13.977195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:13.978004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=6e038ce8a0d447d193a94e48b6eee195, user.id=530d570a2a714da5b2ebc0c36f68bee6) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:13.978307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:13.979733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadac93-08eb-4d5d-bc43-5239a2b1ee06 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:13.982494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:13.985613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 549/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:13 2022] GET /identity/v3/role_assignments?user.id=530d570a2a714da5b2ebc0c36f68bee6&scope.project.id=6e038ce8a0d447d193a94e48b6eee195&role.id=87cd2d147f344fa99848788e1a735d8f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:16.048699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbefb4-334f-442c-9b02-56e097234506 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.049231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbefb4-334f-442c-9b02-56e097234506 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.049231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbefb4-334f-442c-9b02-56e097234506 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.050605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 569/1118] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:16.055692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.056128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.056228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.072389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:16.073548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:16.136686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 550/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:16.159113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbefb4-334f-442c-9b02-56e097234506 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:16.211097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:16.260857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbefb4-334f-442c-9b02-56e097234506 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:16.293106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:16.293933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.294205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.294473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] PATH_INFO: `/v3/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.295947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] RBAC: Authorizing `identity:get_role(role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:16.297293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:16.298629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 570/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:16 2022] GET /identity/v3/roles/87cd2d147f344fa99848788e1a735d8f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:16.304755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:16.399789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430aae-2a3f-4deb-9067-c94fe5ac3039 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:16.426524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:16.432619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:16.433472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.433752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.434025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] PATH_INFO: `/v3/users/530d570a2a714da5b2ebc0c36f68bee6` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.435281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] RBAC: Authorizing `identity:get_user(user_id=530d570a2a714da5b2ebc0c36f68bee6)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:16.436699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:16.438384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 551/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:16 2022] GET /identity/v3/users/530d570a2a714da5b2ebc0c36f68bee6 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:16.444510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:16.540307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dabff-f081-4211-a91c-06d1e7ed812a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:16.559107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:16.573038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:16.573874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.574151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.574418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.575611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:16.577027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fc070-86c0-4f88-add2-2b19df24e04c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:16.578751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 571/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:16 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:16.585355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:16.681914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcf7af-2e00-4585-b0ff-b566ae041b02 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:16.714829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:16.715701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:16.715980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:16.716251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195/users/530d570a2a714da5b2ebc0c36f68bee6/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:16.718408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e038ce8a0d447d193a94e48b6eee195, user_id=530d570a2a714da5b2ebc0c36f68bee6, role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:16.719861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe81-dab6-4c81-828b-35dc2c4dc430 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:16.732167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 552/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:23:16 2022] PUT /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195/users/530d570a2a714da5b2ebc0c36f68bee6/roles/87cd2d147f344fa99848788e1a735d8f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:23:18.783888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9bd9-d20b-4efa-ad3f-2428ca7e3015 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:18.784393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9bd9-d20b-4efa-ad3f-2428ca7e3015 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:18.784393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9bd9-d20b-4efa-ad3f-2428ca7e3015 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:18.786231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 572/1124] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:18.791251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:18.791673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:18.791844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:18.809493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:18.810670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:18.862142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:18.876388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 553/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:18.899924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca9bd9-d20b-4efa-ad3f-2428ca7e3015 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:18.997120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca9bd9-d20b-4efa-ad3f-2428ca7e3015 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:19.029397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:19.030225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:19.030504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:19.030802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] PATH_INFO: `/v3/roles/87cd2d147f344fa99848788e1a735d8f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:19.032264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] RBAC: Authorizing `identity:get_role(role_id=87cd2d147f344fa99848788e1a735d8f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:19.033614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:19.035020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 573/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:18 2022] GET /identity/v3/roles/87cd2d147f344fa99848788e1a735d8f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:23:19.040898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:19.135251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f062-7b19-43bf-b096-2420d47c26f8 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:19.168120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:19.168962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:19.169242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:19.169511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] PATH_INFO: `/v3/users/530d570a2a714da5b2ebc0c36f68bee6` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:19.170678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] RBAC: Authorizing `identity:get_user(user_id=530d570a2a714da5b2ebc0c36f68bee6)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:19.172121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:19.173694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 554/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:19 2022] GET /identity/v3/users/530d570a2a714da5b2ebc0c36f68bee6 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:19.179058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:19.233627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:19.276172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb88a1-816f-4466-aa9b-2a278964b820 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:19.308256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:19.308834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:19.309116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:19.309390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:19.375124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:19.376551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:19.428647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:19.471441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 574/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:19.478471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:19.533228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:19.578107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb9ab2-b68d-4eec-a6f6-73c2ddc6928c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:19.612132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:19.612969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:19.613271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:19.613541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] PATH_INFO: `/v3/projects/6e038ce8a0d447d193a94e48b6eee195` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:19.614696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] RBAC: Authorizing `identity:get_project(project_id=6e038ce8a0d447d193a94e48b6eee195)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:19.616146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcadb06-6ccd-4614-9e5a-63e86c6bc4ef None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:19.617815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 555/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:19 2022] GET /identity/v3/projects/6e038ce8a0d447d193a94e48b6eee195 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:19.624339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:19.718580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533da0a-0348-4b03-b69c-53c57eb4c92f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:19.750929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:19.751558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:19.751830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:19.752100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:19.752914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] RBAC: Adding query filter params (role.id=87cd2d147f344fa99848788e1a735d8f, scope.project.id=6e038ce8a0d447d193a94e48b6eee195, user.id=530d570a2a714da5b2ebc0c36f68bee6) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:19.753217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:19.754656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcb62a-97ac-4d10-a317-083949ae8041 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:19.760289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 575/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:19 2022] GET /identity/v3/role_assignments?user.id=530d570a2a714da5b2ebc0c36f68bee6&scope.project.id=6e038ce8a0d447d193a94e48b6eee195&role.id=87cd2d147f344fa99848788e1a735d8f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:21.855476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8667-a602-4fad-a425-9c60e0893d02 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:21.856015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8667-a602-4fad-a425-9c60e0893d02 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:21.856015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8667-a602-4fad-a425-9c60e0893d02 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:21.857273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 556/1131] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:21.862301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:21.862693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:21.862876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:21.879718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:21.880880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:21.895275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:21.946031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 576/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:21.969043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e8667-a602-4fad-a425-9c60e0893d02 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:22.066990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e8667-a602-4fad-a425-9c60e0893d02 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:22.089617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:22.100625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:22.101654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:22.101934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:22.102205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:22.103393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:22.104819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8e50b-cdf2-463c-9618-b8e517bc161f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:22.106451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 557/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:22.112158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:22.205789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fae1da-0963-4f5c-addd-9149b3b600bd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:22.239150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:22.239793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:22.240072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:22.240346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:22.241046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:22.242469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c2959-b768-4cc3-8e05-0a7e4948e801 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:22.266782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 577/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:23:22 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:23:24.401430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fe954-3e8a-4ad2-9a42-e3a5fce03320 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:24.402000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fe954-3e8a-4ad2-9a42-e3a5fce03320 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:24.402000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fe954-3e8a-4ad2-9a42-e3a5fce03320 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:24.403369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 558/1135] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:24.408575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:24.408965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:24.409125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:24.414071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:24.427407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:24.428567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:24.493905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 578/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:24.518257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fe954-3e8a-4ad2-9a42-e3a5fce03320 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:24.617512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fe954-3e8a-4ad2-9a42-e3a5fce03320 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:24.649536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:24.650299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:24.650581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:24.650884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:24.652108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:24.653482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a50ea3-b942-4a00-a2aa-019fa5d43dfa None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:24.655087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 559/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:24.661191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:24.759535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d38f-9dff-4709-a429-39a25cc029f3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:24.787747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:24.792996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:24.793649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:24.793927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:24.794194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:24.794930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:24.796361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb598-d8a5-4b9f-988b-fb955d8f0c47 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:24.816335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 579/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:23:24 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:23:26.944364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956aded-501d-416f-8f01-1d2d75fe5cdc None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:26.944868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956aded-501d-416f-8f01-1d2d75fe5cdc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:26.944868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956aded-501d-416f-8f01-1d2d75fe5cdc None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:26.946160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 560/1139] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:26.951218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046069-0930-431f-a50a-5ca4090e561e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:26.951610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046069-0930-431f-a50a-5ca4090e561e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:26.951774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71046069-0930-431f-a50a-5ca4090e561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:26.973071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-71046069-0930-431f-a50a-5ca4090e561e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:26.974319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-71046069-0930-431f-a50a-5ca4090e561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:27.040005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 580/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:27.063435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956aded-501d-416f-8f01-1d2d75fe5cdc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:27.121786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:27.162266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956aded-501d-416f-8f01-1d2d75fe5cdc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:27.193853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:27.194671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:27.194931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:27.195200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:27.204841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:27.212790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:27.213777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:27.214726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 561/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:27.219749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71046069-0930-431f-a50a-5ca4090e561e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:27.322514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71046069-0930-431f-a50a-5ca4090e561e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:27.340160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:27.358479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:27.359178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:27.359456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:27.359723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:27.360470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:27.360776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:27.362148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:27.370160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 581/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:27.375942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:27.475378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e83b6-1120-4b2f-bb87-287e4e08d9d0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:27.494513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:27.509340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:27.510193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:27.510471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:27.510739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] PATH_INFO: `/v3/users/ad13d3cda4c944d08aa750cc2f213fcf` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:27.523854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] RBAC: Authorizing `identity:get_user(user_id=ad13d3cda4c944d08aa750cc2f213fcf)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:27.525271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:27.526911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 562/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/users/ad13d3cda4c944d08aa750cc2f213fcf => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:27.532748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:27.630465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2f755-8eb0-4910-9c2c-5aa2c06f866b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:27.662790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:27.663430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:27.663710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:27.663983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:27.710219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:27.729591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:27.730990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:27.824648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 582/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:27.830457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:27.926570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796331d-4617-47da-bc11-e97eb9147530 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:27.959569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:27.960412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:27.960692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:27.960966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] PATH_INFO: `/v3/projects/515c13ba60ca477bac5e351143f59126` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:27.962128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] RBAC: Authorizing `identity:get_project(project_id=515c13ba60ca477bac5e351143f59126)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:27.963586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213f886-5f4f-46db-aa59-2d01a05c37b0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:27.965258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 563/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/projects/515c13ba60ca477bac5e351143f59126 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:27.971372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:28.048661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:28.068372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950c427-f8e6-44b7-b7d8-72121ae811e7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:28.100425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:28.101316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:28.101594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:28.101866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:28.102688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=515c13ba60ca477bac5e351143f59126, user.id=ad13d3cda4c944d08aa750cc2f213fcf) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:28.103025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:28.104446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41b096-8f5a-4437-8d33-32e29ed4054f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:28.109756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 583/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:27 2022] GET /identity/v3/role_assignments?user.id=ad13d3cda4c944d08aa750cc2f213fcf&scope.project.id=515c13ba60ca477bac5e351143f59126&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:30.156419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cf419-f96f-4776-9abf-e45222c9df5b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.156946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cf419-f96f-4776-9abf-e45222c9df5b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.156946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cf419-f96f-4776-9abf-e45222c9df5b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.158310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 564/1147] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:30.163548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280629-7555-46ec-a9e9-472121664684 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.163913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280629-7555-46ec-a9e9-472121664684 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.164098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280629-7555-46ec-a9e9-472121664684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.180919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-98280629-7555-46ec-a9e9-472121664684 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:30.182089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-98280629-7555-46ec-a9e9-472121664684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:30.246446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 584/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:30.268750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cf419-f96f-4776-9abf-e45222c9df5b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:30.314725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:30.367197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cf419-f96f-4776-9abf-e45222c9df5b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:30.399549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:30.400315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.400595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.400866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.410700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:30.418180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:30.419305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:30.420380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 565/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:30.425879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280629-7555-46ec-a9e9-472121664684 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:30.498953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:30.522897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280629-7555-46ec-a9e9-472121664684 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:30.555751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:30.556402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.556678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.556948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.557677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:30.557977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:30.559360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:30.567144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 585/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:30.572968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:30.669173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb832ec1-2079-4b14-b91b-f76ecc3f159f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:30.676308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:30.702405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:30.703275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.703555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.703830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] PATH_INFO: `/v3/users/ad13d3cda4c944d08aa750cc2f213fcf` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.704994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] RBAC: Authorizing `identity:get_user(user_id=ad13d3cda4c944d08aa750cc2f213fcf)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:30.706446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:30.708055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 566/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:30 2022] GET /identity/v3/users/ad13d3cda4c944d08aa750cc2f213fcf => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:30.713611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:30.810824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10c3bd-9a95-457e-bcf8-3cc8461ca2ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:30.842605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:30.843475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.843752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.844020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] PATH_INFO: `/v3/projects/515c13ba60ca477bac5e351143f59126` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.845171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] RBAC: Authorizing `identity:get_project(project_id=515c13ba60ca477bac5e351143f59126)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:30.846592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fda47-08f5-402d-b15c-3bc802ba5850 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:30.848902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 586/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:30 2022] GET /identity/v3/projects/515c13ba60ca477bac5e351143f59126 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:30.855001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:30.948667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9efac-12ad-49e5-884f-1649e9bb74da None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:30.981597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:30.982454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:30.982730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:30.983025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] PATH_INFO: `/v3/projects/515c13ba60ca477bac5e351143f59126/users/ad13d3cda4c944d08aa750cc2f213fcf/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:30.984924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=515c13ba60ca477bac5e351143f59126, user_id=ad13d3cda4c944d08aa750cc2f213fcf, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:30.986328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497694d6-0d9e-4fa7-9772-d7e34e3160c2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:31.000133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 567/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:23:30 2022] PUT /identity/v3/projects/515c13ba60ca477bac5e351143f59126/users/ad13d3cda4c944d08aa750cc2f213fcf/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:23:33.077143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ec70-f797-4847-b276-bccd30df2ffd None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.077707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ec70-f797-4847-b276-bccd30df2ffd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.077707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ec70-f797-4847-b276-bccd30df2ffd None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.079103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 587/1154] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:33.084778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.085165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.085327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.103069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:33.104244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:33.129564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:33.169667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 568/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:33.192440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5ec70-f797-4847-b276-bccd30df2ffd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:33.196228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:33.292177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5ec70-f797-4847-b276-bccd30df2ffd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:33.323599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:33.324371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.324645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.324913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.335688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:33.343109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:33.344195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:33.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 588/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:33.351098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:33.444993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b902ad-9f9d-45c3-87d1-091c685937bd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:33.478442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:33.479124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.479401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.479670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.480403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:33.480705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:33.482059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:33.490313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 569/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:33.495855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:33.556359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:33.593415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c293f5-3691-4dd1-b60e-53d4dc506670 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:33.625944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:33.626816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.627102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.627377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] PATH_INFO: `/v3/users/ad13d3cda4c944d08aa750cc2f213fcf` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.629082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] RBAC: Authorizing `identity:get_user(user_id=ad13d3cda4c944d08aa750cc2f213fcf)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:33.630504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:33.632135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 589/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:33 2022] GET /identity/v3/users/ad13d3cda4c944d08aa750cc2f213fcf => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:33.637713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:33.664717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:33.733933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993d9a-518d-473a-9888-22b22d91340d None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:33.765653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:33.766281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:33.766557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:33.766853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:33.830532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:33.832561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:33.857034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:33.926512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 570/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:33.932213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:34.027523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64eee3-4b64-47a0-b8b7-7862f1b94e84 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:34.055876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:34.061322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:34.062169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:34.062448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:34.062720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] PATH_INFO: `/v3/projects/515c13ba60ca477bac5e351143f59126` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:34.063910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] RBAC: Authorizing `identity:get_project(project_id=515c13ba60ca477bac5e351143f59126)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:34.065335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66985f-1ebe-43e9-8d97-2d1179b87724 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:34.067652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 590/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:33 2022] GET /identity/v3/projects/515c13ba60ca477bac5e351143f59126 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:34.073585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:34.170006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b80144-84ca-43a8-8015-e6276a20de59 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:34.195027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:34.203659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:34.204290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:34.204567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:34.204834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:34.205646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=515c13ba60ca477bac5e351143f59126, user.id=ad13d3cda4c944d08aa750cc2f213fcf) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:34.205950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:34.207407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1c6c2-ecff-4ddc-b6d4-744952afaa6a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:34.212998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 571/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:34 2022] GET /identity/v3/role_assignments?user.id=ad13d3cda4c944d08aa750cc2f213fcf&scope.project.id=515c13ba60ca477bac5e351143f59126&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:36.302879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634586-81f0-4eef-a57d-6b225d12a061 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:36.303411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634586-81f0-4eef-a57d-6b225d12a061 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:36.303411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634586-81f0-4eef-a57d-6b225d12a061 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:36.304784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 591/1162] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:36.309950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:36.310327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:36.310498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:36.327423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:36.328592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:36.393087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 572/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:36.415193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634586-81f0-4eef-a57d-6b225d12a061 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:36.515099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634586-81f0-4eef-a57d-6b225d12a061 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:36.548611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:36.549237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:36.549513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:36.550363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:36.611694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:36.614471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:36.616217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:36.708927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 592/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:36.714784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:36.810630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a573c2-054f-49d3-80e2-c71afe7ca240 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:36.830393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:36.845085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:36.845874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:36.846147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:36.846411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:36.853837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:36.855289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c1c0d6e3-83ff-4a89-a927-5bd706a44fc7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:36.869332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:36.870531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:36.871597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 573/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:23:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:36.876393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:36.968274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:36.971250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fac23-1779-4b15-aa8d-395f5b4d32da None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:37.004679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:37.005320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:37.005600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:37.005873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:37.006586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:37.006921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:37.008283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dfd4d-6786-4486-a9a7-acb95e5cb8d5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:37.015262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 593/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Wed Jun 15 00:23:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:39.081801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907daa-fd99-481a-a0ad-53c7d5267dce None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:39.082358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907daa-fd99-481a-a0ad-53c7d5267dce None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:39.082358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907daa-fd99-481a-a0ad-53c7d5267dce None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:39.083731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 574/1167] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:39.089506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834f21e-7306-463e-9eb2-1fdec9ab8d12 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:39.089919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834f21e-7306-463e-9eb2-1fdec9ab8d12 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:39.090063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834f21e-7306-463e-9eb2-1fdec9ab8d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:39.106635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e834f21e-7306-463e-9eb2-1fdec9ab8d12 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:39.107868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e834f21e-7306-463e-9eb2-1fdec9ab8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:39.172051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 594/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:39.195290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74907daa-fd99-481a-a0ad-53c7d5267dce None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:39.293466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74907daa-fd99-481a-a0ad-53c7d5267dce None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:39.326315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:39.326979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:39.327263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:39.327536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:39.328261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:39.329616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e5612-260a-4b35-9148-fa0c5e8bd737 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:39.343336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 575/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Wed Jun 15 00:23:39 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:23:41.455863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605303b-f85a-4bb1-a714-e58878996408 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:41.456437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605303b-f85a-4bb1-a714-e58878996408 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:41.456437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605303b-f85a-4bb1-a714-e58878996408 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:41.457777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 595/1170] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:41.462873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:41.463263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:41.463429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:41.480057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:41.481222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:41.515293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:41.547292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 576/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:41.571444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605303b-f85a-4bb1-a714-e58878996408 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:41.661758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:41.669516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605303b-f85a-4bb1-a714-e58878996408 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:41.702792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:41.703659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:41.703938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:41.704208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] PATH_INFO: `/v3/domains/db12ce7ee5e848d1a4346559a23ae84d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:41.714361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=db12ce7ee5e848d1a4346559a23ae84d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:41.715766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b91abb-046a-49b9-b4d0-2fe0e73419ca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:41.717385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 596/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Wed Jun 15 00:23:41 2022] GET /identity/v3/domains/db12ce7ee5e848d1a4346559a23ae84d => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:41.723709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:41.818714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8044a6-816b-428b-bf95-7e44ffd4cfce None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:41.851461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:41.852099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:41.852378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:41.852651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:41.853347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:41.854757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07506e6-ffbe-498b-a439-d384ae51e915 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:41.879013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 577/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:23:41 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:23:43.995391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03233bb-6f02-410c-be79-c8aba0f57261 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:43.995921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03233bb-6f02-410c-be79-c8aba0f57261 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:43.995921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03233bb-6f02-410c-be79-c8aba0f57261 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:43.997291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 597/1174] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:44.002385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:44.003369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:44.003525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:44.020317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:44.021481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:44.037354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:44.086647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 578/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:23:44.111360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03233bb-6f02-410c-be79-c8aba0f57261 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:44.210281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03233bb-6f02-410c-be79-c8aba0f57261 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:44.241880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:44.242717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:44.243020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:44.243287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] PATH_INFO: `/v3/domains/db12ce7ee5e848d1a4346559a23ae84d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:44.244448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=db12ce7ee5e848d1a4346559a23ae84d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:44.246020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581054be-4881-432f-baf5-41a53c8db727 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:44.247847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 598/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Wed Jun 15 00:23:44 2022] GET /identity/v3/domains/db12ce7ee5e848d1a4346559a23ae84d => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:44.254035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:44.350146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d9e40-b87c-4d09-8421-01524410c210 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:44.383773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:44.384420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:44.384700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:44.384973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:44.385679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:44.387135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f49227-3af1-49b4-9279-2861d73f36dc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:44.405554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 579/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:23:44 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:23:46.513457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fbd0-34d8-419f-824e-30ace133574d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:46.513876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fbd0-34d8-419f-824e-30ace133574d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:46.514000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fbd0-34d8-419f-824e-30ace133574d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:46.515405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 599/1178] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:46.521070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:46.521441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:46.521622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:46.537967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:46.539182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:46.541094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:46.608851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 580/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:46.632701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fbd0-34d8-419f-824e-30ace133574d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:46.657268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:46.731962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fbd0-34d8-419f-824e-30ace133574d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:46.764564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:46.765284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:46.765563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:46.765832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:46.776033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:46.783527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:46.784637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:46.785888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 600/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:46.790519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:46.886207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7baec5-e81a-420c-bf2d-33340d1cc18a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:46.917762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:46.918410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:46.918685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:46.918979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:46.919714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:46.920014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:46.921368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:46.926678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:46.930130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 581/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:46.935974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:47.013986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:47.031481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7816c-7325-4c1c-b344-b5687354ab55 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:47.064656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:47.065509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:47.065786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:47.066053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] PATH_INFO: `/v3/users/42deebd653c34e7d95a9e25e1f5c3684` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:47.078527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=42deebd653c34e7d95a9e25e1f5c3684)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:47.080005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:47.081622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 601/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:46 2022] GET /identity/v3/users/42deebd653c34e7d95a9e25e1f5c3684 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:47.087277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:47.183372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4da2-15f1-481c-a920-34fe915c3f13 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:47.214940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:47.215560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:47.215838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:47.216110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:47.281006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:47.283387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:47.285670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:47.380222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 582/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:47.386129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:47.483799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4f0a-f2fe-469c-9fa3-530d338aa7b7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:47.516354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:47.517202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:47.517489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:47.517759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] PATH_INFO: `/v3/projects/88def5a568614426810096839d389964` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:47.519473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] RBAC: Authorizing `identity:get_project(project_id=88def5a568614426810096839d389964)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:47.520891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b9ec2-8718-44a5-ba76-67baf02191d0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:47.522569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 602/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:47 2022] GET /identity/v3/projects/88def5a568614426810096839d389964 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:47.528511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:47.621063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:47.624144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a688b-2dfb-46e3-a8e2-0a2056575736 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:47.657460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:47.658090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:47.658369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:47.659266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:47.660086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=88def5a568614426810096839d389964, user.id=42deebd653c34e7d95a9e25e1f5c3684) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:47.660392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:47.661889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee84632-b91a-45f5-b5b4-8b3497e2522e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:47.667165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 583/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:47 2022] GET /identity/v3/role_assignments?user.id=42deebd653c34e7d95a9e25e1f5c3684&scope.project.id=88def5a568614426810096839d389964&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:49.714602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62a37-eebe-41f8-a3ba-dcc89a78498d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:49.715185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62a37-eebe-41f8-a3ba-dcc89a78498d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:49.715185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62a37-eebe-41f8-a3ba-dcc89a78498d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:49.716526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 603/1186] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:49.721707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:49.722089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:49.722254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:49.739339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:49.740523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:49.803857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 584/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:49.825961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62a37-eebe-41f8-a3ba-dcc89a78498d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:49.836498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:49.923582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62a37-eebe-41f8-a3ba-dcc89a78498d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:49.956906 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:49.957680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:49.957960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:49.958232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:49.967996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:49.976353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:49.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:23:49.978500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 604/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:49.983583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:50.072136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:50.079170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c7fd0-143c-4f09-8ac2-53372c8427ee None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:50.112380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:50.113024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:50.113305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:50.113575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:50.114308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:50.114610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:50.115990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:50.123699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 585/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:50.129175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:50.196168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:50.231450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58273b-9575-47f7-9c7f-fd08677e53cc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:50.264275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:50.265124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:50.265400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:50.265670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] PATH_INFO: `/v3/users/42deebd653c34e7d95a9e25e1f5c3684` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:50.266864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] RBAC: Authorizing `identity:get_user(user_id=42deebd653c34e7d95a9e25e1f5c3684)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:50.268304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:50.269913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 605/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:50 2022] GET /identity/v3/users/42deebd653c34e7d95a9e25e1f5c3684 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:50.275652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:50.372739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947510d-5cbe-44b3-bd61-0c9e51089d31 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:50.404576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:50.405402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:50.405678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:50.405947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] PATH_INFO: `/v3/projects/88def5a568614426810096839d389964` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:50.407126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] RBAC: Authorizing `identity:get_project(project_id=88def5a568614426810096839d389964)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:50.409075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996a7c-53e8-4051-94b5-e236e2c1c473 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:50.410715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 586/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:50 2022] GET /identity/v3/projects/88def5a568614426810096839d389964 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:50.416620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:50.513267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea16a8d-25f8-4d36-983e-ff89464285c8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:50.546516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:50.547420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:50.547701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:50.547970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] PATH_INFO: `/v3/projects/88def5a568614426810096839d389964/users/42deebd653c34e7d95a9e25e1f5c3684/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:50.550435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=88def5a568614426810096839d389964, user_id=42deebd653c34e7d95a9e25e1f5c3684, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:50.551898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371fa9d-8a48-4611-9818-16f579bf8d08 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:50.554892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:50.565576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 606/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:23:50 2022] PUT /identity/v3/projects/88def5a568614426810096839d389964/users/42deebd653c34e7d95a9e25e1f5c3684/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:23:52.625471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de2e31-7388-4730-a0e6-db2701cc0f31 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:52.626007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de2e31-7388-4730-a0e6-db2701cc0f31 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:52.626007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de2e31-7388-4730-a0e6-db2701cc0f31 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:52.628003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 587/1193] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:23:52.633082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:52.633515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:52.633632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:52.649756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:23:52.650955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:23:52.714360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 607/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:23:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:23:52.737977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de2e31-7388-4730-a0e6-db2701cc0f31 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:52.763364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:52.837353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de2e31-7388-4730-a0e6-db2701cc0f31 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:52.870457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:52.871255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:52.871534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:52.871805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:52.882204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:23:52.889760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:23:52.890885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:23:52.892079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 588/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:23:52.896851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:52.990676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013599e8-e34f-47be-acbc-139924cf2f95 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:53.014768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:53.024263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:53.024910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:53.025188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:53.025459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:53.026188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:53.026488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:53.027872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:53.036135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 608/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:53.041653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:53.123139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:53.138627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d57cd0-f037-4d20-9de1-7892d985789e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:53.170735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:53.171610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:53.171890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:53.172163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] PATH_INFO: `/v3/users/42deebd653c34e7d95a9e25e1f5c3684` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:53.173325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] RBAC: Authorizing `identity:get_user(user_id=42deebd653c34e7d95a9e25e1f5c3684)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:53.175386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:53.176988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 589/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Wed Jun 15 00:23:53 2022] GET /identity/v3/users/42deebd653c34e7d95a9e25e1f5c3684 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:53.182704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:53.278488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56652d-a2a6-403b-85ad-b56789c905ae None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:53.310909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:53.311541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:53.311823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:53.312096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:53.365171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:53.377551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:53.379118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:53.473789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 609/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:23:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:23:53.479932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:53.576054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db3c1-4f9b-46a2-9b51-08f7117f4607 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:53.608632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:53.609464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:53.609738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:53.610006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] PATH_INFO: `/v3/projects/88def5a568614426810096839d389964` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:53.611171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] RBAC: Authorizing `identity:get_project(project_id=88def5a568614426810096839d389964)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:53.612622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63443128-32de-4a79-8cc7-0633aaa8e501 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:53.614269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 590/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Wed Jun 15 00:23:53 2022] GET /identity/v3/projects/88def5a568614426810096839d389964 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:23:53.620231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:23:53.704687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:23:53.717410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb288395-55c8-44ff-ad3b-ac02f386f7ed None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:23:53.750544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:23:53.751209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:23:53.751489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:23:53.751759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:23:53.752581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=88def5a568614426810096839d389964, user.id=42deebd653c34e7d95a9e25e1f5c3684) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:23:53.753400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:23:53.754872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828f440-99d5-4803-a756-5138f39abda3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:23:53.760551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 610/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:23:53 2022] GET /identity/v3/role_assignments?user.id=42deebd653c34e7d95a9e25e1f5c3684&scope.project.id=88def5a568614426810096839d389964&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:03.604673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:03.758533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=24 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:49.339818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6463b1-d9a2-4137-8786-5db06c0288ef None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:49.340503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6463b1-d9a2-4137-8786-5db06c0288ef None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:49.340503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6463b1-d9a2-4137-8786-5db06c0288ef None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:49.342225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 591/1201] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:24:49.348951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:49.348951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:49.348951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:49.368179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:24:49.370226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:24:49.450816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 611/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 00:24:49.475526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6463b1-d9a2-4137-8786-5db06c0288ef None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:49.593629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6463b1-d9a2-4137-8786-5db06c0288ef None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:49.637284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:49.638387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:49.638899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:49.639339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:49.651728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:49.653246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:24:49.673821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:49.675095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:49.676088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:24:49.676088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:24:49.676088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:49.676088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:24:49.676088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 592/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:49.684060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:49.804443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa997-b855-4d12-9187-40fc14d3f5eb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:49.838541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:49.847519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:49.848156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:49.848646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:49.848912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:49.850129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:49.850396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:49.851855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c054274-dc41-447b-85f9-8b34fb42003f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:49.871018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 612/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:49.872275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:49.949348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:50.012355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1cc45-9620-4f54-b299-a74efaa0233c None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:50.062769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:50.063484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:50.064376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:50.064698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:50.065628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:50.067508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12799466-4935-4c9c-bd18-976ff27dc83a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:50.098909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 593/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Wed Jun 15 00:24:49 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:24:52.536270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3bc10-880f-44d1-b23c-9315178f4812 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:52.536817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3bc10-880f-44d1-b23c-9315178f4812 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:52.537030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3bc10-880f-44d1-b23c-9315178f4812 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:52.539241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 613/1206] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:24:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:24:52.546811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:52.546811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:52.546811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:52.565203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:24:52.566684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:24:52.654905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 594/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:24:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:24:52.678547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3bc10-880f-44d1-b23c-9315178f4812 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:52.797365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3bc10-880f-44d1-b23c-9315178f4812 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:52.836067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:52.837076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:52.837365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:52.837635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:52.849869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:24:52.858152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:52.859159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:52.860039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 614/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:52.866414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:52.908957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:52.980932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd13cc-7d07-49ea-bb66-cd2fd0e0c3fb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:53.020759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:53.021432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:53.021719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:53.021972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:53.022717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:53.023101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:53.024611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:53.036368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 595/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:53.043738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:53.105103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:53.166226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c84f0c-988d-4f9d-92f5-e19dcc7ffe31 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:53.205811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:53.206614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:53.206935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:53.207362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:53.218785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:53.220874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:24:53.241171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:53.242194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:53.243137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:24:53.243137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:24:53.243137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:53.243137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:53.243137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 615/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:24:53.248382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:53.376364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:53.379661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51bd7d-81c6-4a6a-9385-de00521dafbc None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:53.420519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:53.421190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:53.421595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:53.421832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:53.422461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:53.422944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:53.424341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:53.439744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 596/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:53.445480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:53.569846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640adad-01b3-4571-ad6d-6044d7d2e40f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:53.571495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:53.614653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:53.615559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:53.615840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:53.616169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:53.624569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:53.626519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:24:53.643535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:53.644617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:53.645597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 616/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:53.651234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:53.773043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0297-5f98-447b-872a-5b060db5d2fd None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:53.825564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:53.826228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:53.826504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:53.826793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:53.827690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:53.827938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:53.829927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:53.849518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 597/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:24:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:53.855477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:53.976583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3961361-0faf-4b45-bdfe-94e15be05330 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:54.014265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:54.014944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:54.015224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:54.015494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:54.049582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:54.093887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:54.095395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:54.213985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 617/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:24:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:24:54.221862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:54.246044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:54.350397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aa338-7d82-4bb6-b42b-7675344204be None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:54.395293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:54.396150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:54.396439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:54.396706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:54.409513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:54.410987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:24:54.429157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:54.430326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:54.431539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:24:54.431539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:24:54.431539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:54.431539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:24:54.437501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:54.439273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 598/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:24:54.518200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:54.577951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb142-c147-49fe-9afe-9c0c139e46cc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:54.640781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:54.641932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:54.642416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:54.642915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:54.644245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:54.644786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:54.647317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:54.670682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 618/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:24:54.680134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:54.790237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:54.824471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c902e-63d9-41a7-969b-1c028e2eb54e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:54.867515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:54.868450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:54.868818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:54.869117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:54.877449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:54.879542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:24:54.894583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:54.895738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:24:54.896774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 599/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:54.901494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:55.014713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75309e1b-486e-47fb-b223-69dacc3b25f2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:55.039436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:55.053156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:55.053861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:55.054228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:55.054677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:55.056149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:55.056474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:55.058027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa05ec7-2f02-4413-8184-e0e5bbcb0fa0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:55.071920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 619/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:55.077890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:55.192624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17edb09-fb42-49ac-8f57-ded3e14bd3e0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:55.226103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:55.232825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:55.233657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:55.234036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:55.234435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:55.235709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=07f71bb8eebf4ec4999192359dafb28d) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:55.236164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:55.237663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa13565-b683-4707-a223-0f11f8ef0aa2 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:55.251377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 600/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:24:55 2022] GET /identity/v3/role_assignments?user.id=07f71bb8eebf4ec4999192359dafb28d&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:57.452984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80abea-c727-4aa2-b443-ac7e8c1c6613 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:57.453518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80abea-c727-4aa2-b443-ac7e8c1c6613 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:57.453518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80abea-c727-4aa2-b443-ac7e8c1c6613 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:57.455590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 620/1220] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:24:57.461077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:57.461522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:57.461648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:57.481863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:24:57.483090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:24:57.548942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 601/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:24:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:24:57.573130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80abea-c727-4aa2-b443-ac7e8c1c6613 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:57.680042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80abea-c727-4aa2-b443-ac7e8c1c6613 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:57.716570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:57.717392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:57.717643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:57.717944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:57.729051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:24:57.737026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:57.738174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:57.739656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 621/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:57.745784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:57.860870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b7da2-bee2-46ac-8c47-249ebe5a0374 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:57.884318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:57.895202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:57.895867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:57.896146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:57.896416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:57.897158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] RBAC: Adding query filter params (name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:57.897462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:57.898839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:57.907057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 602/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:24:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:57.913745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:57.964291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:58.034178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d7d84-a1d2-4ae5-86e0-6aafe9afd2ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.073183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.074024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.074269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.074592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.085771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:58.087327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:24:58.104506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:58.105651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:58.106699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:24:58.106699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:24:58.106699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:58.106699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:58.106699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 622/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:58.112420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:58.220129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bb66-a00d-4ab3-b750-62135303e735 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.256790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.257445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.257726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.257995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.258715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:58.259055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:58.260419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:58.273796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 603/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:58.281108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:58.391005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:58.398452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9ae9-9dfe-4ad3-860b-1d9bad9ba241 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.435108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.435912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.436196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.436545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.445112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:58.447081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:24:58.464512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:58.466020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:58.467496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 623/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:58.473074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:58.526645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:58.587061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d69b3-3d63-4c00-a6a2-9e459218a661 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.625093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.625784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.626070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.626350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.627410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:58.627744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:58.629460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:58.649820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 604/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:24:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:24:58.655479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:58.765119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d1b-8fd2-4271-a6de-cfe3a4b67b61 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.805070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.806269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.806640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.806967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.820391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:58.822362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:24:58.840991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:58.842073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:58.843170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:24:58.843170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:24:58.843170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:24:58.843170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:58.843170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 624/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:58.849037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:58.961260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860fa50-1996-46fa-8fbd-3588aa2f5279 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:58.963579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:58.997306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:58.997951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:58.998227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:58.998502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:58.999242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:58.999543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:59.001451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:59.012994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 605/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:59.019470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:59.029445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:59.128793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dcc5c-7ce4-41ca-99c3-43e40780d203 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:59.167015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:59.168123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:59.168413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:59.168910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:59.177214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:59.179029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:24:59.197212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:24:59.198315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:24:59.199412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 625/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:24:59.204996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:59.338692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eccc9b-ff40-4a8c-a236-5707b2f62a36 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:59.377184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:59.377883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:59.378223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:59.378501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:59.379312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:24:59.379622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:59.381130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688ee3-c19e-4be7-9625-2b6af525f6b1 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:59.394566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 606/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:24:59.400685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:24:59.472081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:24:59.510411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8b777-5c56-4bca-9fec-c2884181bc84 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:24:59.548773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:24:59.549843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:24:59.550169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:24:59.550602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/07f71bb8eebf4ec4999192359dafb28d/roles/c33254fe3dc24030bb9af26c2ecf94a8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:24:59.566805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=07f71bb8eebf4ec4999192359dafb28d, role_id=c33254fe3dc24030bb9af26c2ecf94a8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:24:59.568847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfd4fc-b01e-41d4-ae6d-c6bdb309d3e9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:24:59.583554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 626/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:24:59 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/07f71bb8eebf4ec4999192359dafb28d/roles/c33254fe3dc24030bb9af26c2ecf94a8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:25:01.987764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17c54c-44a7-4fe3-9285-f13b645c4f4a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:01.988365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17c54c-44a7-4fe3-9285-f13b645c4f4a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:01.988365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17c54c-44a7-4fe3-9285-f13b645c4f4a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:01.990041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 607/1233] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:25:01.996333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:01.996751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:01.996975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:02.017049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:02.018226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:02.087810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 627/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:02.113194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17c54c-44a7-4fe3-9285-f13b645c4f4a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:02.140285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:02.235759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17c54c-44a7-4fe3-9285-f13b645c4f4a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:02.276398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:02.277140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:02.277463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:02.277835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:02.290478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:25:02.303650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:02.304663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:02.305700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 608/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Wed Jun 15 00:25:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:02.311230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:02.365431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:02.427939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff5128-0f4b-4ea5-87d5-a7ac4c6f10a4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:02.465267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:02.466134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:02.466614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:02.467124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:02.468114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:02.468617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:02.471222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:02.482852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 628/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Wed Jun 15 00:25:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:02.488493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:02.583923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:02.603002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723b4a-b1f0-4f0a-bd65-62743cf98b81 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:02.637507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:02.638289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:02.638571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:02.638874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:02.651474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:02.653027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:02.670940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:02.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:02.672957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:02.672957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:02.672957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:02.672957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:02.672957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 609/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 00:25:02.678017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:02.796394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b503-465a-400e-aec9-02a46516b922 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:02.807013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:02.837129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:02.837721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:02.837995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:02.838304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:02.839162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:02.839514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:02.840973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:02.858534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 629/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:02.860186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:02.964953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13725-895d-4370-86a2-705cec97a5b0 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:02.999234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:03.000029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:03.000307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:03.001089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:03.003996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:03.010796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:03.012723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:25:03.028496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:03.029606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:03.030563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 610/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:03.035711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:03.150795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084253c0-e5ee-41d6-9a38-7eff8f10d384 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:03.186639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:03.187378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:03.187633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:03.187964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:03.188842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:03.189153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:03.190597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:03.210086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 630/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:25:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:03.216186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:03.339923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4841dca-a082-47ca-85cf-3680ac1d049b None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:03.377210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:03.377945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:03.378224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:03.378491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:03.444270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:03.451600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:03.453054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:03.560355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 611/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:25:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:25:03.566860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:03.596884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:03.674920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e4681-e956-401a-99b8-70c458cce3b3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:03.711549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:03.712389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:03.712761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:03.713034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:03.723937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:03.725630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:03.743767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:03.744905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:03.750246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:03.750246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:03.750246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:03.750246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:03.750246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 631/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:03.752749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:03.857324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:03.864209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb3ec-f157-4d66-99e0-7d6e46b10c34 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:03.904521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:03.905238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:03.905585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:03.905888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:03.906684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:03.907031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:03.908490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:03.920898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 612/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:25:03.927211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:04.011880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:04.034883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d50ce-d7c7-476a-ad0b-83a2fc0458e3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:04.073172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:04.073975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:04.074239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:04.074516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:04.081338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:04.082799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:25:04.096321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:04.097383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:04.098614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 632/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:04.103545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:04.213762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bab2e-05ef-4bdd-a400-eac3283dcf43 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:04.249348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:04.250008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:04.250345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:04.250557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:04.251383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:04.251698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:04.253129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59fcf3-cdc3-4428-ad42-63cbe8c2569b None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:04.267081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 613/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:04.273942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:04.405346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee6a2b-9640-451f-a2cd-982409b081e5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:04.441326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:04.441994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:04.442272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:04.442548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:04.443450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] RBAC: Adding query filter params (role.id=c33254fe3dc24030bb9af26c2ecf94a8, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=07f71bb8eebf4ec4999192359dafb28d) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:04.443760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:04.445181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec72ec-e98f-4fd9-a5fc-4137f466d46b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:04.451389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 633/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:25:04 2022] GET /identity/v3/role_assignments?user.id=07f71bb8eebf4ec4999192359dafb28d&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=c33254fe3dc24030bb9af26c2ecf94a8 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:06.860430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906305-ad2c-43d3-b9fa-afc7bb9db4ca None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:06.860962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906305-ad2c-43d3-b9fa-afc7bb9db4ca None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:06.860962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906305-ad2c-43d3-b9fa-afc7bb9db4ca None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:06.862423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 614/1247] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:06.868848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:06.869285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:06.869399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:06.874592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:06.888062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:06.889281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:06.963013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 634/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:06.988103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84906305-ad2c-43d3-b9fa-afc7bb9db4ca None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:07.009788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:07.103265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84906305-ad2c-43d3-b9fa-afc7bb9db4ca None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:07.144165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:07.144920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:07.145222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:07.145504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:07.158182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:25:07.166936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:07.168065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:07.168943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 615/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:07.174846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:07.301268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5f921-f35b-457c-b142-659a376035ce None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:07.345522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:07.351996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:07.352669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:07.352949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:07.353271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:07.354026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:07.354336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:07.355768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:07.364943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 635/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:07.371696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:07.462240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:07.492576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e87bd-3564-40e2-883a-712345af3de7 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:07.532473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203541-d636-4581-8b35-27d922a69152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:07.533423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203541-d636-4581-8b35-27d922a69152 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:07.533763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203541-d636-4581-8b35-27d922a69152 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:07.534051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203541-d636-4581-8b35-27d922a69152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:07.549547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203541-d636-4581-8b35-27d922a69152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:07.551369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203541-d636-4581-8b35-27d922a69152 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-16203541-d636-4581-8b35-27d922a69152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:07.572339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:07.573344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:07.574322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:07.574322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:07.574322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:07.574322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:07.574322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 616/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:07.580222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:07.701245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a9f12-386d-4deb-999b-919ec13e290c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:07.737358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:07.738016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:07.738297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:07.738567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:07.739394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:07.739697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:07.741066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:07.753379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 636/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:07.759179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203541-d636-4581-8b35-27d922a69152 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:07.864024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203541-d636-4581-8b35-27d922a69152 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:07.894173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:07.904228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:07.905158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:07.905648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:07.905852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:07.915071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:07.916880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:25:07.931880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:07.933033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:07.939389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 617/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:07.939997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:08.000776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:08.050259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1cbb2-7fb5-4899-8151-f2641a512974 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:08.090504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:08.091447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:08.091845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:08.092194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:08.093312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:08.093746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:08.095398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:08.112907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 637/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:25:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:08.118698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:08.229185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a859e-2da5-4771-b8dd-9ac4217d71fa None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:08.269527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:08.270114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:08.270394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:08.270681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:08.336498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:08.351456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:08.353075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:08.458584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 618/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:25:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 00:25:08.466922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:08.576046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7177a-9daf-42de-96b1-389a28a3ea26 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:08.590450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:08.615449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:08.616238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:08.616514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:08.616774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:08.628134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:08.629591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:08.647419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:08.648507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:08.649412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:08.649412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:08.649412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:08.649412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:08.649412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 638/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:08.660458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:08.754477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:08.767165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98642ea-5b84-470f-809c-5857a80eda42 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:08.804773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:08.805441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:08.805722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:08.805994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:08.806733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:08.807069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:08.808468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:08.820981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 619/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:08.826615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:08.930752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9e400-96c8-4f8c-b261-acce991a9129 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:08.965511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:08.966309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:08.966584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:08.966894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:08.974999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:08.976511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:25:08.991683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:08.992832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:08.993913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 639/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:08.999054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:09.102651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee9373-9c9a-4e53-87a5-0907d91fc512 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:09.136379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:09.137038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:09.137328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:09.137601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:09.138388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:09.138726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:09.140213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c92be6-1fda-4b22-ba01-1d6a2794f3c3 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:09.156033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 620/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:09.162848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:09.190498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:09.274050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca79230-f806-45ab-951b-d020106c5fcb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:09.318004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:09.318724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:09.319193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:09.319490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:09.320567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=07f71bb8eebf4ec4999192359dafb28d) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:09.320914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:09.322912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e4427-13fa-49a4-9745-43acfa921b48 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:09.332719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 640/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:25:09 2022] GET /identity/v3/role_assignments?user.id=07f71bb8eebf4ec4999192359dafb28d&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:11.384816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6f68-4ac7-4790-9cb0-8c0e5f42980b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:11.385376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6f68-4ac7-4790-9cb0-8c0e5f42980b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:11.385376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6f68-4ac7-4790-9cb0-8c0e5f42980b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:11.386700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 621/1261] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:11.392362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:11.392797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:11.392890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:11.409674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:11.410874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:11.476339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 641/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:11.498096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6f68-4ac7-4790-9cb0-8c0e5f42980b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:11.508753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:11.596923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6f68-4ac7-4790-9cb0-8c0e5f42980b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:11.628793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:11.629561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:11.629837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:11.630106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:11.639823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:25:11.647788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:11.648949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:11.649872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 622/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:11.655103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:11.673155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:11.749726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5684d0-9bb7-402c-baa0-408e0bef6bcc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:11.782612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:11.783331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:11.783617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:11.783895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:11.784644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:11.784954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:11.786347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:11.794382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 642/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:11.799973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:11.864335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:11.897142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b564cf-5c78-4636-8b3e-e97b541362b4 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:11.928552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:11.929307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:11.929585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:11.929859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:11.940501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:11.941946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:11.959024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:11.960046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:11.961189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:11.961189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:11.961189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:11.961189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:11.961189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 623/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:11.965715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.044273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:12.063793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4ce5f-b1c6-4d0d-823e-2df83e4ffbab None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.095524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.096164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.096450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.096728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.097447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:12.097756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.099407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.111556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 643/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:12.117019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.214035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa7f10-a4e8-478d-bf38-cbfc9c99893e None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.232981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:12.247739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.248515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.248794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.249071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.257816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.259317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:25:12.274227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:12.275330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:12.276310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 624/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:12.281501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.383729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854dd9-1448-4ac7-b522-9e4fb509429a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.416465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.417124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.417405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.417681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.418507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:12.418853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.420271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.429178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:12.437836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 644/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:12.443467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.540139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89891fa-10b3-47b3-abdf-d6ebcccddf61 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.576079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.576864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.577146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.577423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.588945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.591122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:12.612862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:12.613911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:12.614987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:12.614987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:12.614987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:12.614987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:12.614987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 625/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:12.621101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.716758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf97372-29c8-4f66-bf48-08c6ea212b9b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.750020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.750602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.750913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.751185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.751892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:12.752193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.753534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.764279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 645/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:12.769683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.800977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:12.868106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a34ba-41b3-467a-be60-6849957d418f None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:12.900350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:12.901125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:12.901405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:12.901677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:12.908777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:12.910192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:25:12.924171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:12.925204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:12.926405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 626/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:12.931057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:12.987546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:13.027661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfaafe-d63d-408d-bd68-938b5017f86f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:13.059463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:13.060099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:13.060373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:13.060639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:13.061413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:13.061720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:13.063181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea55cd-e9a9-4577-a665-9876ee146e24 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:13.074616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 646/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:13.080922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:13.169587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:13.178367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29c46b-e38c-4e68-83e3-90a62d421d6a None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:13.211326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:13.212185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:13.212464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:13.212732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] PATH_INFO: `/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/07f71bb8eebf4ec4999192359dafb28d/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:13.214629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=491c878c8c7a4e489d2c511d19eb11e5, user_id=07f71bb8eebf4ec4999192359dafb28d, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:13.216068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17856495-5ba3-4101-bb8a-2bfec99471b7 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:13.228431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 627/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Wed Jun 15 00:25:13 2022] PUT /identity/v3/projects/491c878c8c7a4e489d2c511d19eb11e5/users/07f71bb8eebf4ec4999192359dafb28d/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:25:15.350721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614054b-063e-4051-ae14-6cef2aced1fc None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:15.351283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614054b-063e-4051-ae14-6cef2aced1fc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:15.351404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614054b-063e-4051-ae14-6cef2aced1fc None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:15.352920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 647/1274] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:15.358062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:15.358503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:15.358606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:15.375041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:15.376230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:15.439810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 628/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:15.463068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614054b-063e-4051-ae14-6cef2aced1fc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:15.563167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614054b-063e-4051-ae14-6cef2aced1fc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:15.580513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:15.598166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:15.599232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:15.599508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:15.599772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:15.609777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 00:25:15.617261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:15.618351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:15.619586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 648/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Wed Jun 15 00:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:15.624387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:15.701815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:15.720452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e76c5-983b-4003-85d4-a06f3db61d14 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:15.753519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:15.754156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:15.754431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:15.754696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:15.755454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:15.755752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:15.757109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:15.764750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 629/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Wed Jun 15 00:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:15.770934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:15.865722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578055-8552-4760-8ba3-077935f1e52d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:15.897427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:15.898192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:15.898469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:15.898739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:15.908764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:15.910180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:15.927010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:15.928035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:15.928997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:15.928997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:15.928997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:15.928997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:15.928997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 649/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:15.933625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.030070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734cdfa-c073-4498-ab7d-18c90b8ac190 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:16.062301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:16.062969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:16.063251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:16.063525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:16.064236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:16.064540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:16.065889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:16.077070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 630/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:16.083324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.113389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:16.180202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1ffbd-2553-4c11-986e-d306c4fd3e75 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:16.212992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:16.213768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:16.214048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:16.214319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:16.222269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:16.223989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 00:25:16.237896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:16.239139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:16.240328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 650/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:16.246984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.251248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:16.358613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec820b0-e05a-4cab-afb1-2d60cd446d77 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:16.391741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:16.392399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:16.392678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:16.392950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:16.393783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:16.394087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:16.395552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:16.412552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 631/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Wed Jun 15 00:25:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:16.418370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.506132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:16.515872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde011f9-1c1e-410d-88d0-c4ad46b4e5d6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:16.548701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:16.549340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:16.549621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:16.549890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:16.614794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:16.616242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:16.699983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:16.710392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 651/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Wed Jun 15 00:25:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 00:25:16.716155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.775729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:16.813084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fe77a-0791-4b3b-827f-4f80503cae54 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:16.846196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:16.846992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:16.847278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:16.847552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:16.857498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:16.858957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 00:25:16.875731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:16.876637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:16.877587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 00:25:16.877587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 00:25:16.877587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 00:25:16.877587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:16.877587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 632/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:16.882504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:16.978964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e170eb-03c9-4af0-8c7b-b351e25b8a39 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:17.011344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:17.011984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:17.012262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:17.012538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:17.013242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:17.013544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:17.014920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:17.026288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 652/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Wed Jun 15 00:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:17.031791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:17.127189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a9d19-929d-4840-9dba-420b41df5681 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:17.140985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:17.160588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:17.161348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:17.161624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:17.161899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:17.168333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:17.169762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 00:25:17.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:17.184153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: ERROR keystone.server.flask.application Jun 15 00:25:17.185284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 633/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Wed Jun 15 00:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:17.190572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:17.238974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:17.291233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1403cd-2fae-4e25-9f08-b2e1f2d28142 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:17.324207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:17.324855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:17.325137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:17.325411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:17.326201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:17.326509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:17.327966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1e32f-3cbf-45b2-af63-4dd5a8e0ab08 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:17.339620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 653/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Wed Jun 15 00:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:17.345655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:17.442126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078429e-3a0c-4432-92dd-d0bf592b6d35 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:17.473229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:17.474116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:17.474395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:17.474666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:17.475514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] RBAC: Adding query filter params (role.id=8d88da41d3264c6da23235143e5a2f57, scope.project.id=491c878c8c7a4e489d2c511d19eb11e5, user.id=07f71bb8eebf4ec4999192359dafb28d) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:17.475824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:17.477237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cfa5c-cbb2-4825-9afc-6fde8492f491 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:17.483305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 634/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Wed Jun 15 00:25:17 2022] GET /identity/v3/role_assignments?user.id=07f71bb8eebf4ec4999192359dafb28d&scope.project.id=491c878c8c7a4e489d2c511d19eb11e5&role.id=8d88da41d3264c6da23235143e5a2f57 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:19.596290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb016b6-8b55-4938-b97a-4359006a5dd7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:19.596805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb016b6-8b55-4938-b97a-4359006a5dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:19.596896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb016b6-8b55-4938-b97a-4359006a5dd7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:19.598224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 654/1288] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:19.603693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:19.604109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:19.604243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:19.620343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:19.621503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:19.649794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:19.686396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 635/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:19.708511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb016b6-8b55-4938-b97a-4359006a5dd7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:19.806789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb016b6-8b55-4938-b97a-4359006a5dd7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:19.817614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:19.840615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:19.841395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:19.841671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:19.841939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:19.842580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:19.844559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:19.856492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:19.857728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:19.858936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 655/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:25:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:19.863580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:19.960369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8b4ff-fcf2-41c7-bbb5-ce51fdfd5a79 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:19.992475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:19.993121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:19.993401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:19.993673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:19.994367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:19.994670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:19.996053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516d5a-dd58-4246-936b-4817e659e492 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:20.001393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 636/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Wed Jun 15 00:25:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:20.007169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:20.102344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed249aa-c688-4c4a-91af-a40c350ef882 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:20.133959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:20.134591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:20.134895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:20.135165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:20.135858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:20.136161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:20.137495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b60c-b701-4963-ba94-eed9eec68b52 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:20.142314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 656/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Wed Jun 15 00:25:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:22.242386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56511e-86c9-4b69-bc3f-72e43e2e8205 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:22.242941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56511e-86c9-4b69-bc3f-72e43e2e8205 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:22.243061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56511e-86c9-4b69-bc3f-72e43e2e8205 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:22.244330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 637/1293] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:22.249372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1baaf3-8660-409c-bec4-add84bd42682 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:22.250041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1baaf3-8660-409c-bec4-add84bd42682 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:22.250324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1baaf3-8660-409c-bec4-add84bd42682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:22.267109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bd1baaf3-8660-409c-bec4-add84bd42682 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:22.268308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bd1baaf3-8660-409c-bec4-add84bd42682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:22.301892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:22.345808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 657/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:22.368920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56511e-86c9-4b69-bc3f-72e43e2e8205 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:22.398790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:22.467243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56511e-86c9-4b69-bc3f-72e43e2e8205 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:22.503087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:22.503737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:22.504015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:22.504283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:22.504989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:22.506339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ab3f1-4021-48be-a518-652e122fbcdd None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:22.518735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 638/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Wed Jun 15 00:25:22 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:25:24.596041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58d5e-460f-4e3c-ba8a-38f536f1aebe None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:24.596594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58d5e-460f-4e3c-ba8a-38f536f1aebe None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:24.596594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58d5e-460f-4e3c-ba8a-38f536f1aebe None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:24.597961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 658/1296] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:24.603291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:24.603702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:24.603837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:24.620669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:24.621831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:24.690522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 639/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:24.712502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f58d5e-460f-4e3c-ba8a-38f536f1aebe None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:24.812020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f58d5e-460f-4e3c-ba8a-38f536f1aebe None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:24.845069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:24.845853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:24.846133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:24.846404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:24.847085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:24.848697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:24.860411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:24.861625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:24.862496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 659/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:25:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:24.867545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:24.885115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:24.962769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b796fe7-5bc2-4dff-b8fc-455a2c8d72e2 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:24.995445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:24.996083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:24.996361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:24.996631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:24.997320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:24.997621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:24.998992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:25.003986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 640/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Wed Jun 15 00:25:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:25.009777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:25.041225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:25.107952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3224a0-3bc1-434d-9598-d59e326117ae None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:25.139748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:25.140869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:25.141152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:25.141428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:25.142212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] RBAC: Adding query filter params (interface=public, service_id=dbc92bb63e2548e3b4905604308335c4, region_id=RegionOne) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:25.142525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:25.143915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdf79e-5ffa-49e3-b93c-5af33ba2bade None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:25.148835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 660/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Wed Jun 15 00:25:25 2022] GET /identity/v3/endpoints?service_id=dbc92bb63e2548e3b4905604308335c4&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:25.154267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:25.234070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:25.251848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766af0-3b74-4516-8735-7b0b8f8e7749 None admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:25.285089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:25.285731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:25.286034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:25.286304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:25.287008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:25.288351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f6811-400e-4beb-ba40-f825f614195e None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:25.294947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 641/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Wed Jun 15 00:25:25 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:25:27.385299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744931fc-635d-4dd2-ac32-07ad1cd6dc2a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:27.385863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744931fc-635d-4dd2-ac32-07ad1cd6dc2a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:27.385863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744931fc-635d-4dd2-ac32-07ad1cd6dc2a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:27.387245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 661/1302] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:25:27.392789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:27.393208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:27.393344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:27.410314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:25:27.411526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:25:27.475574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 642/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:25:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 00:25:27.498703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744931fc-635d-4dd2-ac32-07ad1cd6dc2a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:27.590617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:27.597903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744931fc-635d-4dd2-ac32-07ad1cd6dc2a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:27.630111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:27.630914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:27.631195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:27.631464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:27.632103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:27.633446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 00:25:27.645578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 00:25:27.646821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Jun 15 00:25:27.648009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 662/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Wed Jun 15 00:25:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 00:25:27.652548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:27.743633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:27.772629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e781-e787-4587-894f-bff0cb2683ee None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:27.810458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:27.811146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:27.811425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:27.812254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] PATH_INFO: `/v3/services` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:27.812950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:25:27.813254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:27.814616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eece4c7-3086-4398-b5f8-95761aea4aed None admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:27.827581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 643/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Wed Jun 15 00:25:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:25:27.828593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:25:27.943883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a137-ae0a-4ec6-abb2-66fc28fb1340 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:25:27.980540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:25:27.981202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:25:27.981485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:25:27.981759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:25:27.982499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:25:27.984473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21f497-f6b2-4290-b015-6f39e981c6a7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:25:28.007971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 663/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Wed Jun 15 00:25:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:25:37.817268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=30 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:25:37.998384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:30.638634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ad6c8-e867-4435-af31-4b5481177c11 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:30.639119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ad6c8-e867-4435-af31-4b5481177c11 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:30.639217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ad6c8-e867-4435-af31-4b5481177c11 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:30.640730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 644/1307] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:30.646793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3963b-8487-4985-b03c-a706fea07f85 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:30.649340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3963b-8487-4985-b03c-a706fea07f85 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:30.649861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3963b-8487-4985-b03c-a706fea07f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:30.737520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2de3963b-8487-4985-b03c-a706fea07f85 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:30.739429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2de3963b-8487-4985-b03c-a706fea07f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:30.861457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 664/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:30.911569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:30.911982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:30.912321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:30.913775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 645/1309] 10.222.0.33 () {54 vars in 964 bytes} [Wed Jun 15 00:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:30.920340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d977ac-c031-44f9-911f-a86581389341 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:30.920630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d977ac-c031-44f9-911f-a86581389341 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:30.920906 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d977ac-c031-44f9-911f-a86581389341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:31.005758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-69d977ac-c031-44f9-911f-a86581389341 None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:31.007030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69d977ac-c031-44f9-911f-a86581389341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:31.028673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:31.088787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 665/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Wed Jun 15 00:26:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:31.097817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:31.099563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:31.212645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7c86-1f2d-480d-acd8-01847836e8a4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:31.291757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:31.292323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:31.292668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:31.293444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:31.295110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:31.352005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:31.415414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:31.416845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3323f-0555-4c04-8c8a-2edbe372abe7 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:31.438964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 646/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:33.887233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:33.887796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:33.887796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:33.889314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 666/1312] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:33.894301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816627-4385-4d99-8a66-50a0e7c34dde None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:33.894716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816627-4385-4d99-8a66-50a0e7c34dde None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:33.894928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816627-4385-4d99-8a66-50a0e7c34dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:33.980692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4a816627-4385-4d99-8a66-50a0e7c34dde None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:33.981921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4a816627-4385-4d99-8a66-50a0e7c34dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:34.091937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 647/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:34.128223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:34.129777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:34.240876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8909-c9bc-4a8b-8dd9-1469bbdabc8d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:34.317234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:34.320174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:34.320774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:34.321051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:34.321339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:34.323255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:34.433919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:34.435419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9390333-9dd6-47cd-b4a0-76ee69187437 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:34.457298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 667/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:36.857686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59479-adb7-46de-99fb-668027cddc13 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:36.858193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59479-adb7-46de-99fb-668027cddc13 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:36.858193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59479-adb7-46de-99fb-668027cddc13 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:36.859538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 648/1315] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:36.864696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f211-8cc0-4161-b6d4-a01d0eece073 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:36.865104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f211-8cc0-4161-b6d4-a01d0eece073 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:36.865255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f211-8cc0-4161-b6d4-a01d0eece073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:36.928648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a033f211-8cc0-4161-b6d4-a01d0eece073 None None] MFA Rules not processed for user `d7a6b76f2705487dafd371ca25ccca47`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:36.929854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a033f211-8cc0-4161-b6d4-a01d0eece073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:36.980729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:37.026237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 668/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:37.080278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:37.080597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:37.080877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:37.082239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 649/1317] 10.222.0.33 () {54 vars in 964 bytes} [Wed Jun 15 00:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:37.087986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1355923-16f7-4a10-b0b7-7ab6d6137100 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:37.088273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1355923-16f7-4a10-b0b7-7ab6d6137100 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:37.088548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1355923-16f7-4a10-b0b7-7ab6d6137100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:37.161560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d1355923-16f7-4a10-b0b7-7ab6d6137100 None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:37.162697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d1355923-16f7-4a10-b0b7-7ab6d6137100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:37.236794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 669/1318] 10.222.0.33 () {58 vars in 1061 bytes} [Wed Jun 15 00:26:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:37.246382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:37.247995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:37.257578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:37.354705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6d43a-9678-427d-b358-0104cb523a2f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:37.426231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:37.426817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:37.427101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:37.427371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:37.428994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:37.489136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:37.537009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:37.538430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf0745-847f-4b75-a9de-925fc3d587b5 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:37.556669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 650/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:40.536403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:40.536924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:40.536924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:40.538316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 670/1320] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:40.543875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124a69-398b-497b-acf7-6e396c9a0db6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:40.544252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124a69-398b-497b-acf7-6e396c9a0db6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:40.544423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124a69-398b-497b-acf7-6e396c9a0db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:40.606409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-77124a69-398b-497b-acf7-6e396c9a0db6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:40.607629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-77124a69-398b-497b-acf7-6e396c9a0db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:40.699396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 651/1321] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:40.737867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:40.739446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:40.788762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:40.852440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7acd07-482d-4848-a18f-f9667c1eb604 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:40.925632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:40.926271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:40.926550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:40.926855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:40.928488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:41.020867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:41.035031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:41.036454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef1c8-7976-49c7-b1b0-9292b7e69418 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:41.054398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 671/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:43.861866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:43.862482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:43.862482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:43.863718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 652/1323] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:43.868732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746eaa2-d47e-4c0c-9013-00dbfa5e7d67 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:43.869140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746eaa2-d47e-4c0c-9013-00dbfa5e7d67 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:43.869320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746eaa2-d47e-4c0c-9013-00dbfa5e7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:43.941489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a746eaa2-d47e-4c0c-9013-00dbfa5e7d67 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:43.942707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a746eaa2-d47e-4c0c-9013-00dbfa5e7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:44.045664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 672/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:44.082561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:44.084141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:44.103211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:44.186349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a67329-651b-4be7-a356-2a2f895154b5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:44.258901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:44.259543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:44.259820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:44.260090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:44.262325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:44.333315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:44.372148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:44.373588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a318cd-ae67-46fc-841f-dfa1aa6b247b service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:44.391597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 653/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:46.570597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:46.571195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:46.571195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:46.572561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 673/1326] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:46.577900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c252a-cd79-40bf-806c-7e44710c32d7 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:46.578346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c252a-cd79-40bf-806c-7e44710c32d7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:46.578450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c252a-cd79-40bf-806c-7e44710c32d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:46.641225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-813c252a-cd79-40bf-806c-7e44710c32d7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:46.642419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-813c252a-cd79-40bf-806c-7e44710c32d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:46.736989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 654/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:46.772494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:46.774039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:46.835812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:46.886993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cae770-f252-4853-8e92-2fc1a745c645 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:46.962292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:46.962969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:46.963253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:46.963527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:46.965695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:47.078826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:47.081430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:47.083586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0943b5-9106-496c-bbda-cec3a3132413 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:47.102634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 674/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:49.286279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:49.286859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:49.286859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:49.288228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 655/1329] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:49.293312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29c0b-acff-4c75-9686-65e9ac6cc96d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:49.293744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29c0b-acff-4c75-9686-65e9ac6cc96d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:49.293876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29c0b-acff-4c75-9686-65e9ac6cc96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:49.358262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-89b29c0b-acff-4c75-9686-65e9ac6cc96d None None] MFA Rules not processed for user `d7a6b76f2705487dafd371ca25ccca47`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:49.359496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-89b29c0b-acff-4c75-9686-65e9ac6cc96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:49.458077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 675/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:49.495568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:49.497108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:49.528135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:49.616087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e956-6505-4b30-9201-15554f72ce63 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:49.694646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:49.695337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:49.695618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:49.695890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:49.697526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:49.773693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:49.804912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:49.806405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d37905-fc0a-4aa6-88ed-c76f9a3ab6ce service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:49.825374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 656/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:52.201923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:52.202487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:52.202487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:52.203900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 676/1332] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:52.209216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767895a7-1fcb-4e92-b30e-78a5ed85cd42 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:52.209619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767895a7-1fcb-4e92-b30e-78a5ed85cd42 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:52.209769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767895a7-1fcb-4e92-b30e-78a5ed85cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:52.273201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-767895a7-1fcb-4e92-b30e-78a5ed85cd42 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:52.274418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-767895a7-1fcb-4e92-b30e-78a5ed85cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:52.368458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 657/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:52.404682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:52.406207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:52.481680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:52.516816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af64dc9-b820-4bb4-ba85-c24e456d472e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:52.588895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:52.589783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:52.590266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:52.590549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:52.592230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:52.703243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:52.704675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bd2d1-f7bd-4a49-bb11-32db08a96e2d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:52.714992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:52.723874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 677/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:54.937266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:54.937773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:54.937773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:54.939127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 658/1335] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:54.944208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d6d6-98e6-4a4e-8c1a-b59b2b0a2058 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:54.944655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d6d6-98e6-4a4e-8c1a-b59b2b0a2058 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:54.944757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d6d6-98e6-4a4e-8c1a-b59b2b0a2058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:55.007925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3a52d6d6-98e6-4a4e-8c1a-b59b2b0a2058 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:55.009119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3a52d6d6-98e6-4a4e-8c1a-b59b2b0a2058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:55.104649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 678/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:55.144257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:55.145785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:55.185604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:55.247936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89523-5398-4807-8823-4a6159f8d3c1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:55.325455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:55.326091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:55.326370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:55.326639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:55.328312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:55.409369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:55.434668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:55.436129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218be7b4-ad62-4518-9a26-803fdf7494d0 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:55.455534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 659/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:26:57.864142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:57.864648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:57.864648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:57.866029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 679/1338] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:26:57.871864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd63ec-70ad-4814-a5bf-426592c897cf None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:57.872250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd63ec-70ad-4814-a5bf-426592c897cf None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:57.872412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd63ec-70ad-4814-a5bf-426592c897cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:57.935707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0ccd63ec-70ad-4814-a5bf-426592c897cf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:26:57.936922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0ccd63ec-70ad-4814-a5bf-426592c897cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:58.030724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 660/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:26:58.066422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:26:58.067960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:58.142268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:26:58.172167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385d156-f5bc-42e4-a0e0-299c2b3a2452 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:26:58.242576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:26:58.243229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:26:58.243507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:26:58.243776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:26:58.245423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:26:58.356851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:26:58.358277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b62fb-32ae-4816-9514-6654586f08bc service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:26:58.375700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 680/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:26:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:02.364303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:02.364784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:02.364875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:02.366573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 661/1341] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:02.372845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040757-19f2-4bc0-be4d-5e6a5f828477 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:02.373331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040757-19f2-4bc0-be4d-5e6a5f828477 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:02.373423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040757-19f2-4bc0-be4d-5e6a5f828477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:02.379616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:02.445743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5c040757-19f2-4bc0-be4d-5e6a5f828477 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:02.447011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5c040757-19f2-4bc0-be4d-5e6a5f828477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:02.554954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 681/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:02.595417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:02.597908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:02.649832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:02.697586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6a65-cf43-4e1c-aa3a-2ba47a77d2ea None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:02.701176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:02.701813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:02.702093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:02.702364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:02.711459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:02.713437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32207733-b63d-4dc4-a51f-5f4b98d1b888 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:02.723185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 662/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:05.475499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6cca-1903-4cb8-b1e5-99cb6e459fb7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:05.476020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6cca-1903-4cb8-b1e5-99cb6e459fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:05.476105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6cca-1903-4cb8-b1e5-99cb6e459fb7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:05.477526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 682/1344] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:05.484025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709f61-870d-406e-9dcf-f4464b5c7c7c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:05.484548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709f61-870d-406e-9dcf-f4464b5c7c7c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:05.484665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13709f61-870d-406e-9dcf-f4464b5c7c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:05.514194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-13709f61-870d-406e-9dcf-f4464b5c7c7c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:05.515450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-13709f61-870d-406e-9dcf-f4464b5c7c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:05.597323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 663/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:05.638770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6cca-1903-4cb8-b1e5-99cb6e459fb7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:05.649310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6cca-1903-4cb8-b1e5-99cb6e459fb7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:05.653315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:05.653948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:05.654229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:05.654500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:05.663394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:05.664829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6af40-d656-471e-a67e-54c4934b7501 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:05.675228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 683/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:09.699953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723c39-2ae7-4909-8f0b-2be18437a2dd None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:09.700467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723c39-2ae7-4909-8f0b-2be18437a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:09.700595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723c39-2ae7-4909-8f0b-2be18437a2dd None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:09.702295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 664/1347] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:09.707579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014661eb-86a1-4ff5-8010-de9962ff79ea None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:09.708054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014661eb-86a1-4ff5-8010-de9962ff79ea None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:09.708154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014661eb-86a1-4ff5-8010-de9962ff79ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:09.730089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-014661eb-86a1-4ff5-8010-de9962ff79ea None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:09.731341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-014661eb-86a1-4ff5-8010-de9962ff79ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:09.739615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 684/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:09.776277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62723c39-2ae7-4909-8f0b-2be18437a2dd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:09.786734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62723c39-2ae7-4909-8f0b-2be18437a2dd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:09.792520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:09.793166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:09.793442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:09.793719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:09.803080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:09.804546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a0e3-874f-4a62-905d-6f74a82c56be service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:09.814120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 665/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:14.815042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cedbb-6e2c-4620-974d-9c543666d145 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:14.816055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cedbb-6e2c-4620-974d-9c543666d145 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:14.816540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cedbb-6e2c-4620-974d-9c543666d145 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:14.818424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 685/1350] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:27:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:14.823272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289822c8-2621-4f18-ad45-e58170b2c1f8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:14.823686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289822c8-2621-4f18-ad45-e58170b2c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:14.823864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289822c8-2621-4f18-ad45-e58170b2c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:14.895445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-289822c8-2621-4f18-ad45-e58170b2c1f8 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:14.896620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-289822c8-2621-4f18-ad45-e58170b2c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:14.936162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:15.008229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 666/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:27:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:19.220984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78ab3-5323-464e-b1a8-5bc5fc379763 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:19.221497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78ab3-5323-464e-b1a8-5bc5fc379763 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:19.221497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78ab3-5323-464e-b1a8-5bc5fc379763 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:19.223087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 686/1352] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:19.228722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1876a70-aebf-4c14-a2eb-4ee53c74fde7 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:19.229717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1876a70-aebf-4c14-a2eb-4ee53c74fde7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:19.230323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1876a70-aebf-4c14-a2eb-4ee53c74fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:19.250486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c1876a70-aebf-4c14-a2eb-4ee53c74fde7 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:19.251754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c1876a70-aebf-4c14-a2eb-4ee53c74fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:19.261400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 667/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:19.724016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=72 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:20.721019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ea15-f51d-4b23-bc24-ed4b8655ac19 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:20.721637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ea15-f51d-4b23-bc24-ed4b8655ac19 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:20.721637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ea15-f51d-4b23-bc24-ed4b8655ac19 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:20.723024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 687/1354] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:20.728776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664481-df85-4273-9925-1b499387a9e8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:20.729234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664481-df85-4273-9925-1b499387a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:20.729348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664481-df85-4273-9925-1b499387a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:20.747116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-11664481-df85-4273-9925-1b499387a9e8 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:20.748233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-11664481-df85-4273-9925-1b499387a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:20.756670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 668/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:27:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:24.625268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ee844-c812-41b4-bbcd-7d3b9eac97e3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:24.625800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ee844-c812-41b4-bbcd-7d3b9eac97e3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:24.625800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ee844-c812-41b4-bbcd-7d3b9eac97e3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:24.627310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 688/1356] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:24.632677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ac9e-9854-48e5-888f-18d1de8e7c16 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:24.633106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ac9e-9854-48e5-888f-18d1de8e7c16 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:24.633237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ac9e-9854-48e5-888f-18d1de8e7c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:24.653815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-21c3ac9e-9854-48e5-888f-18d1de8e7c16 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:24.655103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-21c3ac9e-9854-48e5-888f-18d1de8e7c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:24.665795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 669/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:24.718364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554694-db28-48b7-a658-e6192685f00e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:24.718651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554694-db28-48b7-a658-e6192685f00e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:24.718958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff554694-db28-48b7-a658-e6192685f00e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:24.720463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 689/1358] 10.222.0.33 () {54 vars in 961 bytes} [Wed Jun 15 00:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:24.728774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef45f9-19b6-4dba-b2db-32c8816c9a09 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:24.729326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef45f9-19b6-4dba-b2db-32c8816c9a09 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:24.729827 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef45f9-19b6-4dba-b2db-32c8816c9a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:24.786843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d6ef45f9-19b6-4dba-b2db-32c8816c9a09 None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:24.788030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d6ef45f9-19b6-4dba-b2db-32c8816c9a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:24.875767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 670/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Wed Jun 15 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:24.884280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff554694-db28-48b7-a658-e6192685f00e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:24.899950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff554694-db28-48b7-a658-e6192685f00e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:24.905713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:24.906626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:24.907125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:24.907595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:24.916856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:24.918663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cff2ac-7442-41b8-8655-7005452746ce service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:24.929109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 690/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:27.276673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a550-264f-4416-93df-45a5133c2836 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:27.277255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a550-264f-4416-93df-45a5133c2836 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:27.277255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a550-264f-4416-93df-45a5133c2836 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:27.278813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 671/1361] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:27.284521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74e04-271a-480f-b5e4-fa7d1c3e4cb8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:27.285104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74e04-271a-480f-b5e4-fa7d1c3e4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:27.285340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74e04-271a-480f-b5e4-fa7d1c3e4cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:27.308281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-30d74e04-271a-480f-b5e4-fa7d1c3e4cb8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:27.309607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-30d74e04-271a-480f-b5e4-fa7d1c3e4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:27.319695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 691/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:27.376163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a550-264f-4416-93df-45a5133c2836 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:27.390606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a550-264f-4416-93df-45a5133c2836 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:27.396251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:27.396890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:27.397173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:27.397453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:27.407314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:27.408916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa815c-5327-47cd-b9d1-fcb62d347223 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:27.419445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 672/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:29.891672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04eac-adef-424a-9923-9359eba5bde2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:29.892185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04eac-adef-424a-9923-9359eba5bde2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:29.892305 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04eac-adef-424a-9923-9359eba5bde2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:29.893633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 692/1364] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:29.898902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b622-fac6-43a0-9952-92f320607d44 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:29.899289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b622-fac6-43a0-9952-92f320607d44 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:29.899470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b622-fac6-43a0-9952-92f320607d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:29.919757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-da07b622-fac6-43a0-9952-92f320607d44 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:29.920973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da07b622-fac6-43a0-9952-92f320607d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:29.929048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 673/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:29.974963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649c7a-63e5-4dc3-b933-d61461869786 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:29.975249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649c7a-63e5-4dc3-b933-d61461869786 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:29.975521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649c7a-63e5-4dc3-b933-d61461869786 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:29.976892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 693/1366] 10.222.0.33 () {54 vars in 961 bytes} [Wed Jun 15 00:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:29.982544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2fb2-91af-4c02-9f38-df23cc45b375 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:29.982855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2fb2-91af-4c02-9f38-df23cc45b375 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:29.983135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2fb2-91af-4c02-9f38-df23cc45b375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:30.005274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8aef2fb2-91af-4c02-9f38-df23cc45b375 None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:30.005274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8aef2fb2-91af-4c02-9f38-df23cc45b375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:30.013725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 674/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Wed Jun 15 00:27:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:30.022689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb649c7a-63e5-4dc3-b933-d61461869786 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:30.037131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb649c7a-63e5-4dc3-b933-d61461869786 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:30.043284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:30.044010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:30.044328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:30.044638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:30.055007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:30.057193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb430c-944b-497a-9611-16ee9555303c service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:30.068146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 694/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:32.280380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73d3d-e2a9-410f-9821-00c0bc994c60 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:32.281496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73d3d-e2a9-410f-9821-00c0bc994c60 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:32.281856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73d3d-e2a9-410f-9821-00c0bc994c60 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:32.283513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 675/1369] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:32.288417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773f11-88be-4381-941a-25f8be8a54cf None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:32.288816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773f11-88be-4381-941a-25f8be8a54cf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:32.288976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773f11-88be-4381-941a-25f8be8a54cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:32.307413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-26773f11-88be-4381-941a-25f8be8a54cf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:32.308612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-26773f11-88be-4381-941a-25f8be8a54cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:32.316934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 695/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:32.365554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73d3d-e2a9-410f-9821-00c0bc994c60 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:32.385447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73d3d-e2a9-410f-9821-00c0bc994c60 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:32.391042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:32.391695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:32.391981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:32.392258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:32.402079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:32.405002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a0469-d64c-4840-a3c2-2ad18cd4fa16 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:32.417479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 676/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:32.764243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e19c-10f4-4bff-98a6-1d8fe9c99cf2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:32.764527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e19c-10f4-4bff-98a6-1d8fe9c99cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:32.764999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e19c-10f4-4bff-98a6-1d8fe9c99cf2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:32.766532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 696/1372] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:32.771802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556ef26-f8e7-4688-8e45-ffe97ff2a14a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:32.772094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556ef26-f8e7-4688-8e45-ffe97ff2a14a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:32.772369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556ef26-f8e7-4688-8e45-ffe97ff2a14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:32.791454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9556ef26-f8e7-4688-8e45-ffe97ff2a14a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:32.792647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9556ef26-f8e7-4688-8e45-ffe97ff2a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:32.800769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 677/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:33.100590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031cf66-0ce8-4c15-88e4-5ef93bcc0bc1 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:33.100858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031cf66-0ce8-4c15-88e4-5ef93bcc0bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:33.101138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031cf66-0ce8-4c15-88e4-5ef93bcc0bc1 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:33.102564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 697/1374] 10.222.0.33 () {54 vars in 963 bytes} [Wed Jun 15 00:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:33.107754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26ba2-7da6-4a53-89fa-c94efb4b24c0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:33.108586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26ba2-7da6-4a53-89fa-c94efb4b24c0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:33.108586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26ba2-7da6-4a53-89fa-c94efb4b24c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:33.116244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:33.159905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-53a26ba2-7da6-4a53-89fa-c94efb4b24c0 None None] MFA Rules not processed for user `7fb0a7dcacc745229341c1c02c575721`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:33.161194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-53a26ba2-7da6-4a53-89fa-c94efb4b24c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:33.224636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 678/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Wed Jun 15 00:27:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:33.232024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031cf66-0ce8-4c15-88e4-5ef93bcc0bc1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:33.243135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031cf66-0ce8-4c15-88e4-5ef93bcc0bc1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:33.246769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:33.247443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:33.247727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:33.248001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:33.258819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:33.260263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d03329-6e82-4ef0-a3c8-296a07300180 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:33.270594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 698/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:34.648008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a952-35f6-4820-a07b-c39673478681 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:34.648479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a952-35f6-4820-a07b-c39673478681 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:34.648590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a952-35f6-4820-a07b-c39673478681 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:34.650029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 679/1377] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:34.655367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb1f8-adb8-4653-b1bf-696c301dc264 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:34.655756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb1f8-adb8-4653-b1bf-696c301dc264 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:34.655954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb1f8-adb8-4653-b1bf-696c301dc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:34.676651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a35eb1f8-adb8-4653-b1bf-696c301dc264 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:34.677846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a35eb1f8-adb8-4653-b1bf-696c301dc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:34.686183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 699/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:34.730157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a952-35f6-4820-a07b-c39673478681 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:34.745040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a952-35f6-4820-a07b-c39673478681 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:34.750503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:34.751175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:34.751459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:34.751729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:34.760308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:34.761883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642155c-61f6-4783-9f7e-3ac7da6fbd01 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:34.775013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 680/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:36.929816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88cf8-3511-4923-99c8-0e6ec394603f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:36.930332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88cf8-3511-4923-99c8-0e6ec394603f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:36.930332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88cf8-3511-4923-99c8-0e6ec394603f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:36.931683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 700/1380] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:36.936821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0ba8-219b-4cb7-af66-b7bfd28b8b71 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:36.937253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0ba8-219b-4cb7-af66-b7bfd28b8b71 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:36.937352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0ba8-219b-4cb7-af66-b7bfd28b8b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:36.956221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-27ae0ba8-219b-4cb7-af66-b7bfd28b8b71 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:36.957392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-27ae0ba8-219b-4cb7-af66-b7bfd28b8b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:36.965891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 681/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:37.011724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88cf8-3511-4923-99c8-0e6ec394603f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:37.030700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88cf8-3511-4923-99c8-0e6ec394603f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:37.037707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:37.038351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:37.038629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:37.038933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:37.047648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:37.049225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d323b09-4f1f-42ae-9957-3bf4833d078f service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:37.059183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 701/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:39.207821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b496c0-49ac-413a-b805-9ae2f715cc25 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:39.208415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b496c0-49ac-413a-b805-9ae2f715cc25 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:39.208415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b496c0-49ac-413a-b805-9ae2f715cc25 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:39.209865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 682/1383] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:39.215718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767050c-d2e0-413b-a962-2c9c636025c4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:39.216143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767050c-d2e0-413b-a962-2c9c636025c4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:39.216260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767050c-d2e0-413b-a962-2c9c636025c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:39.234199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5767050c-d2e0-413b-a962-2c9c636025c4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:39.235403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5767050c-d2e0-413b-a962-2c9c636025c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:39.243559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 702/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:39.288433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b496c0-49ac-413a-b805-9ae2f715cc25 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:39.302685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b496c0-49ac-413a-b805-9ae2f715cc25 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:39.308182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:39.308817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:39.309096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:39.309364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:39.318183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:39.319786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dd488-75cd-428a-bb27-3abcccf7ab29 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:39.330221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 683/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:41.463281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63bb5e-9004-40d6-a22c-4e079dda09c9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:41.463871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63bb5e-9004-40d6-a22c-4e079dda09c9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:41.463871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63bb5e-9004-40d6-a22c-4e079dda09c9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:41.465693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 703/1386] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:41.471595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a51ab-f1cf-4fa6-8cab-1f2315a94346 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:41.472079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a51ab-f1cf-4fa6-8cab-1f2315a94346 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:41.472289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a51ab-f1cf-4fa6-8cab-1f2315a94346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:41.491457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-955a51ab-f1cf-4fa6-8cab-1f2315a94346 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:41.493249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-955a51ab-f1cf-4fa6-8cab-1f2315a94346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:41.501371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 684/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:41.545318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63bb5e-9004-40d6-a22c-4e079dda09c9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:41.560184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63bb5e-9004-40d6-a22c-4e079dda09c9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:41.565594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:41.566230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:41.566513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:41.566795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:41.575298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:41.576889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003c817-6d2a-4321-9fb9-c8b6d368f2a8 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:41.586386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 704/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:44.164296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7335f-a93d-41b6-b753-a1ed285929fc None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:44.164980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7335f-a93d-41b6-b753-a1ed285929fc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:44.165139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7335f-a93d-41b6-b753-a1ed285929fc None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:44.167592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 685/1389] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:44.174074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7a3a-9df2-4627-9894-695ba9f1a05f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:44.174552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7a3a-9df2-4627-9894-695ba9f1a05f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:44.174660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7a3a-9df2-4627-9894-695ba9f1a05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:44.193578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6c5c7a3a-9df2-4627-9894-695ba9f1a05f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:44.195011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6c5c7a3a-9df2-4627-9894-695ba9f1a05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:44.204823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 705/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:44.252131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7335f-a93d-41b6-b753-a1ed285929fc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:44.267057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7335f-a93d-41b6-b753-a1ed285929fc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:44.272951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:44.274241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:44.274569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:44.275002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:44.294243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:44.297670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f56cf-9236-471f-a0e0-26547fcb14a7 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:44.314458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 686/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:27:46.494351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef0df-5d21-48f6-a3bd-5ba1476ade89 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:46.494858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef0df-5d21-48f6-a3bd-5ba1476ade89 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:46.494962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef0df-5d21-48f6-a3bd-5ba1476ade89 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:46.496333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 706/1392] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:46.501692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77837-9181-437e-984b-438bb79db77a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:46.502135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77837-9181-437e-984b-438bb79db77a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:46.502242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77837-9181-437e-984b-438bb79db77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:46.520428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-31a77837-9181-437e-984b-438bb79db77a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:46.521624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-31a77837-9181-437e-984b-438bb79db77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:46.530356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 687/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:46.574609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eef0df-5d21-48f6-a3bd-5ba1476ade89 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:46.589934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eef0df-5d21-48f6-a3bd-5ba1476ade89 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:46.595461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:46.596365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:46.596646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:46.596922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:46.605474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:46.607100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563436-5f45-43d9-8407-7ed73e56f67d service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:46.616804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 707/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:48.541286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659e90-9860-40bc-8306-3de324cde3b5 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.541837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659e90-9860-40bc-8306-3de324cde3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.541837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659e90-9860-40bc-8306-3de324cde3b5 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.543276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 688/1395] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:48.548436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142e4d-de6e-46eb-8c03-8d7e0320dc70 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.548857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142e4d-de6e-46eb-8c03-8d7e0320dc70 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.548997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142e4d-de6e-46eb-8c03-8d7e0320dc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.567489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-29142e4d-de6e-46eb-8c03-8d7e0320dc70 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:48.568687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-29142e4d-de6e-46eb-8c03-8d7e0320dc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:48.577307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 708/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:48.632630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae82e-ba8a-4b52-894d-56e26f7f6586 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.632911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae82e-ba8a-4b52-894d-56e26f7f6586 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.633767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae82e-ba8a-4b52-894d-56e26f7f6586 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.635151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 689/1397] 10.222.0.33 () {54 vars in 962 bytes} [Wed Jun 15 00:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:48.640175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.640457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.640731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.692226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] MFA Rules not processed for user `6fe6cdefe4174887b82a53df1e2a9fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:48.693480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:48.747626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce2c59-a73f-458d-83d1-725ec700eccf None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.747998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce2c59-a73f-458d-83d1-725ec700eccf None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.748281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce2c59-a73f-458d-83d1-725ec700eccf None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.749737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 690/1398] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:27:48.755208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.755696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.756074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.757803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 709/1399] 10.222.0.33 () {58 vars in 1059 bytes} [Wed Jun 15 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:48.765903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:48.773411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634eec40-e16f-448d-a5c7-c9f0374aa05f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:48.773825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:48.775166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:48.777399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:48.778032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.778315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.778589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.783062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 691/1400] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:27:48.787918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:48.789881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d52628-740e-415b-a0e9-befe699e30a0 service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:48.799248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 710/1401] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:27:48.826390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:48.839152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273aebd-0c0c-485e-9d15-02e566336360 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:48.844792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:48.845454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:48.845739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:48.846279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:48.856243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:48.857872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32167e-fa28-4763-a90e-36d92843f3ee service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:48.869293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 692/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:27:49.514341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553a20-9c7e-43ef-8d5b-111bd0c88346 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:49.514728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553a20-9c7e-43ef-8d5b-111bd0c88346 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:49.515113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553a20-9c7e-43ef-8d5b-111bd0c88346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:49.522133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:27:49.563642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dd553a20-9c7e-43ef-8d5b-111bd0c88346 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:49.564949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dd553a20-9c7e-43ef-8d5b-111bd0c88346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:49.632896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 711/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:49.674474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b1da-0a65-4927-9533-0b236e6b0a1e None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:49.674984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b1da-0a65-4927-9533-0b236e6b0a1e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:49.675182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b1da-0a65-4927-9533-0b236e6b0a1e None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:49.676683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 693/1404] 10.222.0.33 () {54 vars in 942 bytes} [Wed Jun 15 00:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:49.683323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d491-4374-4264-ace8-ea84d8819dc1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:49.683680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d491-4374-4264-ace8-ea84d8819dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:49.684027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d491-4374-4264-ace8-ea84d8819dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:49.732618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3249d491-4374-4264-ace8-ea84d8819dc1 None None] MFA Rules not processed for user `01bd308fead745c7995c1bcf554cec1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:49.733840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3249d491-4374-4264-ace8-ea84d8819dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:49.803545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 712/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Wed Jun 15 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:49.812955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2b1da-0a65-4927-9533-0b236e6b0a1e None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:49.826733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2b1da-0a65-4927-9533-0b236e6b0a1e None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:49.831583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:49.832213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:49.832495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:49.832772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:49.842615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:49.844491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8801-3752-48b6-91f2-e5e3af0902f4 service swift] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:49.852972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 694/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:27:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:27:51.037022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e11ba-dc6d-4b1a-aea6-7f89e4d785f0 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:51.037652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e11ba-dc6d-4b1a-aea6-7f89e4d785f0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:51.037898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e11ba-dc6d-4b1a-aea6-7f89e4d785f0 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:51.040245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 713/1407] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:27:51.049568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf32b3-0c58-4a5d-8700-dc5961f6d781 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:51.050137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf32b3-0c58-4a5d-8700-dc5961f6d781 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:51.050229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf32b3-0c58-4a5d-8700-dc5961f6d781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:51.074584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-81bf32b3-0c58-4a5d-8700-dc5961f6d781 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:27:51.075853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-81bf32b3-0c58-4a5d-8700-dc5961f6d781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:27:51.090817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 695/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:27:51.127763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e11ba-dc6d-4b1a-aea6-7f89e4d785f0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:27:51.142832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e11ba-dc6d-4b1a-aea6-7f89e4d785f0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:27:51.149431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:27:51.149431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:27:51.151377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:27:51.151377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:27:51.158875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:27:51.160600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f7078-049c-4e5e-aa77-976b108471b0 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:27:51.170326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 714/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:01.068110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=81 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:28:01.165840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:28:12.146242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696422-2c9a-4fef-a770-4211d58a9a3c None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:12.146845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696422-2c9a-4fef-a770-4211d58a9a3c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:12.146845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696422-2c9a-4fef-a770-4211d58a9a3c None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:12.148169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 696/1410] 10.222.0.33 () {54 vars in 918 bytes} [Wed Jun 15 00:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:12.153210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9218a-07d5-4c70-993d-92df66c71849 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:12.153611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9218a-07d5-4c70-993d-92df66c71849 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:12.153759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9218a-07d5-4c70-993d-92df66c71849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:12.171800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aaf9218a-07d5-4c70-993d-92df66c71849 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:12.172985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aaf9218a-07d5-4c70-993d-92df66c71849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:12.182044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 715/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Wed Jun 15 00:28:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:12.196030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e550-b0d9-4e2f-bdf9-3fac84afba07 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:12.196313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e550-b0d9-4e2f-bdf9-3fac84afba07 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:12.196591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e550-b0d9-4e2f-bdf9-3fac84afba07 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:12.197919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 697/1412] 10.222.0.33 () {54 vars in 918 bytes} [Wed Jun 15 00:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:12.202742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d225-d9ef-48e5-ba7a-0504bb3b5edc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:12.203061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d225-d9ef-48e5-ba7a-0504bb3b5edc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:12.203336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d225-d9ef-48e5-ba7a-0504bb3b5edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:12.220649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9b88d225-d9ef-48e5-ba7a-0504bb3b5edc None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:12.222092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9b88d225-d9ef-48e5-ba7a-0504bb3b5edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:12.231613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 716/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Wed Jun 15 00:28:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:14.802411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534485ea-ff66-4a88-be1b-8fc0b4ebe0a6 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:14.802964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534485ea-ff66-4a88-be1b-8fc0b4ebe0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:14.802964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534485ea-ff66-4a88-be1b-8fc0b4ebe0a6 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:14.804285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:14.809549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04323-390d-4be5-89a5-b9856fde0b50 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:14.809952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04323-390d-4be5-89a5-b9856fde0b50 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:14.810108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04323-390d-4be5-89a5-b9856fde0b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:14.827512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5ae04323-390d-4be5-89a5-b9856fde0b50 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:14.828678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5ae04323-390d-4be5-89a5-b9856fde0b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:14.836931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:14.880177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534485ea-ff66-4a88-be1b-8fc0b4ebe0a6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:14.894247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534485ea-ff66-4a88-be1b-8fc0b4ebe0a6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:14.900094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:14.900716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:14.901050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:14.901262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:14.909420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:14.911021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469c6a-d0e1-491a-8a35-5e72a385f9de service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:14.920743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 699/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:15.861497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d54e-5913-481a-8452-580e66e2d7ad None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:15.862027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d54e-5913-481a-8452-580e66e2d7ad None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:15.862027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d54e-5913-481a-8452-580e66e2d7ad None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:15.863394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:15.869011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd14c-5b8c-40f4-8934-42dc9216f2ef None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:15.869409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd14c-5b8c-40f4-8934-42dc9216f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:15.869553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd14c-5b8c-40f4-8934-42dc9216f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:15.886874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-746bd14c-5b8c-40f4-8934-42dc9216f2ef None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:15.888044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-746bd14c-5b8c-40f4-8934-42dc9216f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:15.895649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:16.045165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522d54e-5913-481a-8452-580e66e2d7ad None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:16.060078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522d54e-5913-481a-8452-580e66e2d7ad None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:16.065498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:16.066131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:16.066412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:16.066688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:16.074726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:16.076335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f1661-3ca8-49a4-854d-a24b0c8071ee service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:16.085680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 719/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:17.017009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93b05-5f6c-413f-ab10-50eb819cff99 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:17.017533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93b05-5f6c-413f-ab10-50eb819cff99 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:17.017533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf93b05-5f6c-413f-ab10-50eb819cff99 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:17.018946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:17.023968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbccb8-d8dd-4850-9b66-9d76572531f8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:17.024396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbccb8-d8dd-4850-9b66-9d76572531f8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:17.024541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbccb8-d8dd-4850-9b66-9d76572531f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:17.043004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2ccbccb8-d8dd-4850-9b66-9d76572531f8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:17.044194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2ccbccb8-d8dd-4850-9b66-9d76572531f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:17.052092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:17.202433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf93b05-5f6c-413f-ab10-50eb819cff99 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:17.216451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf93b05-5f6c-413f-ab10-50eb819cff99 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:17.221700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:17.222587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:17.222901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:17.223171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:17.231204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:17.232764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edeb077-db76-4c24-b232-f18a61bee8dc service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:17.241857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 702/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:18.156013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65f71-9859-4ab2-a40e-b2d175b114ed None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:18.156604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65f71-9859-4ab2-a40e-b2d175b114ed None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:18.156604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65f71-9859-4ab2-a40e-b2d175b114ed None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:18.157935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:18.162748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239d73-643c-4dcc-a593-9be215c021cc None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:18.163213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239d73-643c-4dcc-a593-9be215c021cc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:18.163318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239d73-643c-4dcc-a593-9be215c021cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:18.180726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c4239d73-643c-4dcc-a593-9be215c021cc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:18.181892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c4239d73-643c-4dcc-a593-9be215c021cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:18.189471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:18.339665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a65f71-9859-4ab2-a40e-b2d175b114ed None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:18.353943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a65f71-9859-4ab2-a40e-b2d175b114ed None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:18.359278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:18.359907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:18.360184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:18.360457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:18.368539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:18.370131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6d83-7c8c-475a-8511-8f4b7680b2a1 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:18.379418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 722/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:19.298661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf2f2-90df-430f-84f9-0b00c8769e9b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:19.299247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf2f2-90df-430f-84f9-0b00c8769e9b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:19.299247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf2f2-90df-430f-84f9-0b00c8769e9b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:19.300605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:19.305601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640098d5-25d0-4dea-a890-5247b3620ca3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:19.305996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640098d5-25d0-4dea-a890-5247b3620ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:19.306156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640098d5-25d0-4dea-a890-5247b3620ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:19.323735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-640098d5-25d0-4dea-a890-5247b3620ca3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:19.324949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-640098d5-25d0-4dea-a890-5247b3620ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:19.332652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:19.489115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbf2f2-90df-430f-84f9-0b00c8769e9b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:19.507646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbf2f2-90df-430f-84f9-0b00c8769e9b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:19.513603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:19.514238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:19.514519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:19.514874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:19.522823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:19.524403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478183-db6e-452d-81b1-7baddc8397d1 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:19.533552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 705/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:20.373088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab5fe-a08c-42ac-a2b4-8e16769207ba None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:20.373684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab5fe-a08c-42ac-a2b4-8e16769207ba None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:20.373684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab5fe-a08c-42ac-a2b4-8e16769207ba None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:20.375093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:20.379866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35179d-b546-4332-bcca-ff98b1847efa None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:20.380259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35179d-b546-4332-bcca-ff98b1847efa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:20.380419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35179d-b546-4332-bcca-ff98b1847efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:20.398640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8f35179d-b546-4332-bcca-ff98b1847efa None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:20.400474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8f35179d-b546-4332-bcca-ff98b1847efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:20.408458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:20.559009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fab5fe-a08c-42ac-a2b4-8e16769207ba None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:20.573637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fab5fe-a08c-42ac-a2b4-8e16769207ba None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:20.579022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:20.579645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:20.579924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:20.580196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:20.588946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:20.590531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751157bf-bba9-4fb2-ba38-4b8ce53ba73e service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:20.601224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 725/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:21.435169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea91ad-1b23-4fa7-90bf-2f1e956bfa12 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:21.435695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea91ad-1b23-4fa7-90bf-2f1e956bfa12 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:21.435695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea91ad-1b23-4fa7-90bf-2f1e956bfa12 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:21.437021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:21.441854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c712422-3093-460e-9bcb-5c51604fe39c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:21.442234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c712422-3093-460e-9bcb-5c51604fe39c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:21.442406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c712422-3093-460e-9bcb-5c51604fe39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:21.459991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8c712422-3093-460e-9bcb-5c51604fe39c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:21.461180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8c712422-3093-460e-9bcb-5c51604fe39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:21.469331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:21.622835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea91ad-1b23-4fa7-90bf-2f1e956bfa12 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:21.636921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea91ad-1b23-4fa7-90bf-2f1e956bfa12 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:21.642274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:21.642932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:21.643216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:21.643491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:21.651944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:21.653517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb8d12-cdce-48bf-8c11-d72fdd5a144a service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:21.662696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 708/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:22.503850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f602d-aaed-476c-a6d9-73bd845ce871 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:22.504310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f602d-aaed-476c-a6d9-73bd845ce871 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:22.504416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f602d-aaed-476c-a6d9-73bd845ce871 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:22.505765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:22.511102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a580-5846-4d3a-a640-4f11bda2177c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:22.511519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a580-5846-4d3a-a640-4f11bda2177c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:22.511636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a580-5846-4d3a-a640-4f11bda2177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:22.529221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3d29a580-5846-4d3a-a640-4f11bda2177c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:22.530404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3d29a580-5846-4d3a-a640-4f11bda2177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:22.538100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:22.689312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f602d-aaed-476c-a6d9-73bd845ce871 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:22.703230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f602d-aaed-476c-a6d9-73bd845ce871 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:22.708542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:22.709168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:22.709442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:22.709710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:22.717693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:22.719283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7a636-2c20-4340-ba57-98bda415ba1f service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:22.729327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 728/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:23.571527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac5aa-2b3a-4cb1-92c3-44b934c0b55c None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:23.572097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac5aa-2b3a-4cb1-92c3-44b934c0b55c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:23.572097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac5aa-2b3a-4cb1-92c3-44b934c0b55c None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:23.573433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:23.578541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8847-dd57-4444-8a06-87f1cc01d647 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:23.578990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8847-dd57-4444-8a06-87f1cc01d647 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:23.579129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8847-dd57-4444-8a06-87f1cc01d647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:23.597568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-db4a8847-dd57-4444-8a06-87f1cc01d647 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:23.598766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-db4a8847-dd57-4444-8a06-87f1cc01d647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:23.606644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:23.760939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bac5aa-2b3a-4cb1-92c3-44b934c0b55c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:23.774731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bac5aa-2b3a-4cb1-92c3-44b934c0b55c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:23.780424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:23.781045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:23.781322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:23.781594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:23.792110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:23.793783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f5665-e3ac-435e-9ea8-fb081dac4c1f service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:23.803397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 711/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:25.795664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fd79-112d-4d3c-8aec-dfd170998dc9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:25.796198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fd79-112d-4d3c-8aec-dfd170998dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:25.796198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8fd79-112d-4d3c-8aec-dfd170998dc9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:25.797574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:25.802685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfd78-51e9-43d5-b1df-097529e37dd4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:25.803151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfd78-51e9-43d5-b1df-097529e37dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:25.803269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfd78-51e9-43d5-b1df-097529e37dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:25.820495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-732bfd78-51e9-43d5-b1df-097529e37dd4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:25.822198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-732bfd78-51e9-43d5-b1df-097529e37dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:25.829843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:25.867726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8fd79-112d-4d3c-8aec-dfd170998dc9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:25.878336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8fd79-112d-4d3c-8aec-dfd170998dc9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:25.881839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:25.882465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:25.882744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:25.883051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:25.892388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:25.893811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952259e-cb2e-475b-a787-8e02326e1867 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:25.903679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 731/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:28.164223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84a05-d05c-4894-b364-5c20621fb3e2 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:28.164836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84a05-d05c-4894-b364-5c20621fb3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:28.164836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84a05-d05c-4894-b364-5c20621fb3e2 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:28.166199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:28.171523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae809-096c-4c06-8a92-c338bec06d6d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:28.171978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae809-096c-4c06-8a92-c338bec06d6d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:28.172086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae809-096c-4c06-8a92-c338bec06d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:28.189747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-760ae809-096c-4c06-8a92-c338bec06d6d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:28.190944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-760ae809-096c-4c06-8a92-c338bec06d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:28.199174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:28.235962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b84a05-d05c-4894-b364-5c20621fb3e2 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:28.246858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b84a05-d05c-4894-b364-5c20621fb3e2 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:28.250343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:28.250990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:28.251268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:28.251538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:28.260796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:28.262212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab2068-166e-447f-b12d-6cb05139db52 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:28.271284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 714/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:30.676266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9414-a1ef-4886-aa01-05d257570113 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:30.676856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9414-a1ef-4886-aa01-05d257570113 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:30.676856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9414-a1ef-4886-aa01-05d257570113 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:30.678177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:30.683525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4e72-b781-4cdb-924f-8ccd4a28d622 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:30.683966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4e72-b781-4cdb-924f-8ccd4a28d622 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:30.684074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4e72-b781-4cdb-924f-8ccd4a28d622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:30.701446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3f3f4e72-b781-4cdb-924f-8ccd4a28d622 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:30.702629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3f3f4e72-b781-4cdb-924f-8ccd4a28d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:30.710328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:30.748540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c9414-a1ef-4886-aa01-05d257570113 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:30.759064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c9414-a1ef-4886-aa01-05d257570113 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:30.762493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:30.763157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:30.763441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:30.763717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:30.772630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:30.774572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4eca4-2b9c-4afb-a0ce-ae4497534cb0 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:30.784695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 734/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:33.040434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6edc9c-6e1e-4810-b1d4-b20adc41bec3 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:33.040999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6edc9c-6e1e-4810-b1d4-b20adc41bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:33.040999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6edc9c-6e1e-4810-b1d4-b20adc41bec3 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:33.042339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:33.047498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3272da-0b86-4c6d-bd7c-304eda6653bf None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:33.047896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3272da-0b86-4c6d-bd7c-304eda6653bf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:33.048070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3272da-0b86-4c6d-bd7c-304eda6653bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:33.065975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ff3272da-0b86-4c6d-bd7c-304eda6653bf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:33.067203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ff3272da-0b86-4c6d-bd7c-304eda6653bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:33.074964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:33.112984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6edc9c-6e1e-4810-b1d4-b20adc41bec3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:33.124221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6edc9c-6e1e-4810-b1d4-b20adc41bec3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:33.127808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:33.128454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:33.128739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:33.129018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:33.137978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:33.139433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe8f9f-78a6-4246-adfa-325433323582 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:33.148649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 717/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:35.363553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9c6c-0731-435f-8f64-b45eaccfa461 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:35.364110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9c6c-0731-435f-8f64-b45eaccfa461 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:35.364110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9c6c-0731-435f-8f64-b45eaccfa461 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:35.365408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:35.371202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b59cd-5494-4b3f-ba90-be7779df5572 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:35.371652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b59cd-5494-4b3f-ba90-be7779df5572 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:35.371757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b59cd-5494-4b3f-ba90-be7779df5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:35.389254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1b5b59cd-5494-4b3f-ba90-be7779df5572 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:35.390474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1b5b59cd-5494-4b3f-ba90-be7779df5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:35.398339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:35.436250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a9c6c-0731-435f-8f64-b45eaccfa461 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:35.446729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a9c6c-0731-435f-8f64-b45eaccfa461 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:35.450211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:35.450917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:35.451201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:35.451474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:35.460143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:35.461561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fe2b8-adcf-4d8f-8654-1a3615a66c99 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:35.470783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 737/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:37.721191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21722171-6619-4e53-849a-ca640e154cb5 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:37.721746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21722171-6619-4e53-849a-ca640e154cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:37.721746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21722171-6619-4e53-849a-ca640e154cb5 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:37.723474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:37.729119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed955-5195-42d3-863f-f7850b89abdc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:37.729523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed955-5195-42d3-863f-f7850b89abdc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:37.729676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed955-5195-42d3-863f-f7850b89abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:37.747533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-163ed955-5195-42d3-863f-f7850b89abdc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:37.748660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-163ed955-5195-42d3-863f-f7850b89abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:37.756367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:37.793423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21722171-6619-4e53-849a-ca640e154cb5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:37.804032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21722171-6619-4e53-849a-ca640e154cb5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:37.808083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:37.808704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:37.808987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:37.809251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:37.817950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:37.819397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b82141-90db-42a9-99b4-876d58e4324a service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:37.828659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 720/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:39.989502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767bc9-7002-4d9c-80c1-c5f86ea1947f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:39.990046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767bc9-7002-4d9c-80c1-c5f86ea1947f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:39.990046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767bc9-7002-4d9c-80c1-c5f86ea1947f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:39.991615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:39.996787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c0f8-33f6-4267-8c05-e6e5718b4fee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:39.997232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c0f8-33f6-4267-8c05-e6e5718b4fee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:39.997365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c0f8-33f6-4267-8c05-e6e5718b4fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:40.015673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5f23c0f8-33f6-4267-8c05-e6e5718b4fee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:40.016858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5f23c0f8-33f6-4267-8c05-e6e5718b4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:40.025406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:28:40.065478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767bc9-7002-4d9c-80c1-c5f86ea1947f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:40.076322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767bc9-7002-4d9c-80c1-c5f86ea1947f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:40.079783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:40.080405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:40.080684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:40.080954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:40.089631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:40.091075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13987158-463c-4e51-a204-00ab2dd653a0 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:40.100625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 740/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:42.241488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afe54-2d2f-4412-a953-7231720dcdd0 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:42.242038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afe54-2d2f-4412-a953-7231720dcdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:42.242038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afe54-2d2f-4412-a953-7231720dcdd0 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:42.243436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:42.248635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa4b3a-88cc-4696-bb00-c0fcf9ba443e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:42.249033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa4b3a-88cc-4696-bb00-c0fcf9ba443e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:42.249193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa4b3a-88cc-4696-bb00-c0fcf9ba443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:42.255281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=79 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:28:42.268333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7baa4b3a-88cc-4696-bb00-c0fcf9ba443e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:42.269527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7baa4b3a-88cc-4696-bb00-c0fcf9ba443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:42.278110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:42.314699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afe54-2d2f-4412-a953-7231720dcdd0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:42.320205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=73 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:28:42.327242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afe54-2d2f-4412-a953-7231720dcdd0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:42.330762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:42.331411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:42.331686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:42.331967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:42.341283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:42.342698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf9033-0f03-42e2-a87f-2e0ae5109798 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:42.352887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 723/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:44.501045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1343-e357-44fe-871a-bee03c40a3f5 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:44.501483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1343-e357-44fe-871a-bee03c40a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:44.501582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e1343-e357-44fe-871a-bee03c40a3f5 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:44.503005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:44.508860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279ec1-96d4-4b5c-9ea6-6d36fb444652 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:44.509268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279ec1-96d4-4b5c-9ea6-6d36fb444652 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:44.509458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279ec1-96d4-4b5c-9ea6-6d36fb444652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:44.527253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-85279ec1-96d4-4b5c-9ea6-6d36fb444652 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:44.528425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-85279ec1-96d4-4b5c-9ea6-6d36fb444652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:44.536156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:28:44.573091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e1343-e357-44fe-871a-bee03c40a3f5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:44.583650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e1343-e357-44fe-871a-bee03c40a3f5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:44.587743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:44.588364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:44.588638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:44.588907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:44.597609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:44.599314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b9b66-3727-41ab-a31e-50e4d908e99c service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:44.608446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 743/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:46.743317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee92f-7249-491f-b426-c26af9dd476e None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:46.743900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee92f-7249-491f-b426-c26af9dd476e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:46.743900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee92f-7249-491f-b426-c26af9dd476e None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:46.745224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:28:46.750365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df82ff-23d4-4b77-9341-18393e4ba94d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:46.750819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df82ff-23d4-4b77-9341-18393e4ba94d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:46.750951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df82ff-23d4-4b77-9341-18393e4ba94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:46.768226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-60df82ff-23d4-4b77-9341-18393e4ba94d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:28:46.769397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-60df82ff-23d4-4b77-9341-18393e4ba94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:28:46.776987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:28:46.813675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaee92f-7249-491f-b426-c26af9dd476e None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:28:46.824096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaee92f-7249-491f-b426-c26af9dd476e None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:28:46.827550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:28:46.828167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:28:46.828443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:28:46.828709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:28:46.838942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:28:46.840389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6e224-4bff-4295-a02b-e36aa945a235 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:28:46.849884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 726/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:28:56.763329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=11 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:28:56.845508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=14 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:29:06.588076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce105-647f-4381-bdb5-350b74efe9da None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:06.588636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce105-647f-4381-bdb5-350b74efe9da None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:06.588636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ce105-647f-4381-bdb5-350b74efe9da None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:06.589997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 745/1471] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:06.595229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ab11-f6dd-454f-8d9d-0e508829a286 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:06.595650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ab11-f6dd-454f-8d9d-0e508829a286 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:06.595751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ab11-f6dd-454f-8d9d-0e508829a286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:06.613974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-40d8ab11-f6dd-454f-8d9d-0e508829a286 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:06.615187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-40d8ab11-f6dd-454f-8d9d-0e508829a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:06.622972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 727/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:06.669216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e834-a7fc-419a-8dcf-1225931ac49e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:06.669504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e834-a7fc-419a-8dcf-1225931ac49e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:06.669784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e834-a7fc-419a-8dcf-1225931ac49e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:06.671211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 746/1473] 10.222.0.33 () {54 vars in 962 bytes} [Wed Jun 15 00:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:06.675922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab544a6-8063-4dd7-aa24-df6ad7b49181 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:06.676221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab544a6-8063-4dd7-aa24-df6ad7b49181 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:06.676499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab544a6-8063-4dd7-aa24-df6ad7b49181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:06.693991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-aab544a6-8063-4dd7-aa24-df6ad7b49181 None None] MFA Rules not processed for user `6fe6cdefe4174887b82a53df1e2a9fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:06.695214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-aab544a6-8063-4dd7-aa24-df6ad7b49181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:06.701535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 728/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Wed Jun 15 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:06.708506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333e834-a7fc-419a-8dcf-1225931ac49e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:06.720332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333e834-a7fc-419a-8dcf-1225931ac49e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:06.724093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:06.724711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:06.724993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:06.725268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:06.734676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:06.736126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607732c-30b0-4fba-8246-31d66a619b0c service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:06.745435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 747/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:08.915340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c426c8-dee1-4891-8266-86c1bbb070e1 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:08.915855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c426c8-dee1-4891-8266-86c1bbb070e1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:08.915855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c426c8-dee1-4891-8266-86c1bbb070e1 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:08.917151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 729/1476] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:08.922275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ecca6-4b93-4d9c-b7d1-dd263e177426 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:08.922767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ecca6-4b93-4d9c-b7d1-dd263e177426 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:08.922928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ecca6-4b93-4d9c-b7d1-dd263e177426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:08.940749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bf3ecca6-4b93-4d9c-b7d1-dd263e177426 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:08.941929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bf3ecca6-4b93-4d9c-b7d1-dd263e177426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:08.950257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 748/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:08.972733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c426c8-dee1-4891-8266-86c1bbb070e1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:08.985037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c426c8-dee1-4891-8266-86c1bbb070e1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:08.989376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:08.989999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:08.990278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:08.990547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:08.991292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:08.993618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:29:08.993618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:29:08.993618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:29:08.994743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089c49e-6de7-479a-a33c-94a6f7eaacfd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:09.013689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 730/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Wed Jun 15 00:29:08 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:29:11.137146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a38bd-0a10-4aef-9b7d-5f729adffe4d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:11.137613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a38bd-0a10-4aef-9b7d-5f729adffe4d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:11.137709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a38bd-0a10-4aef-9b7d-5f729adffe4d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:11.139129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 749/1479] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:11.144364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee162-9fe7-443c-b5c0-0a013ef1f776 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:11.144761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee162-9fe7-443c-b5c0-0a013ef1f776 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:11.144916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee162-9fe7-443c-b5c0-0a013ef1f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:11.162500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-539ee162-9fe7-443c-b5c0-0a013ef1f776 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:11.163858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-539ee162-9fe7-443c-b5c0-0a013ef1f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:11.171743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 731/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:29:11.214248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a38bd-0a10-4aef-9b7d-5f729adffe4d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:11.228934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a38bd-0a10-4aef-9b7d-5f729adffe4d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:11.234216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:11.234880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:11.235160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:11.235429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:11.243861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:11.245493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b73872-d88f-4aa2-b382-abb7eb00e79a service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:11.254746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 750/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:13.529951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186e78-d835-4076-8479-27bbb894ff37 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:13.530391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186e78-d835-4076-8479-27bbb894ff37 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:13.530500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186e78-d835-4076-8479-27bbb894ff37 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:13.531897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 732/1482] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:13.536984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f161b-d92c-4f9a-95a3-767f4f9398a9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:13.537421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f161b-d92c-4f9a-95a3-767f4f9398a9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:13.537530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f161b-d92c-4f9a-95a3-767f4f9398a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:13.556206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-077f161b-d92c-4f9a-95a3-767f4f9398a9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:13.557453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-077f161b-d92c-4f9a-95a3-767f4f9398a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:13.565248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 751/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:13.600628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05186e78-d835-4076-8479-27bbb894ff37 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:13.612185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05186e78-d835-4076-8479-27bbb894ff37 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:13.615697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:13.616324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:13.616608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:13.616884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:13.626249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:13.627709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24adb8a-34fc-4a87-b87f-cdaaf26fc0e2 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:13.636992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 733/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:15.800452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16737661-6c1e-430e-9dcf-fcd864087975 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:15.800965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16737661-6c1e-430e-9dcf-fcd864087975 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:15.800965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16737661-6c1e-430e-9dcf-fcd864087975 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:15.802313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 752/1485] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:15.808221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b300e2-f7d2-4c75-b76e-964c5b630cb4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:15.808672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b300e2-f7d2-4c75-b76e-964c5b630cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:15.808768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b300e2-f7d2-4c75-b76e-964c5b630cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:15.826021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-70b300e2-f7d2-4c75-b76e-964c5b630cb4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:15.827244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-70b300e2-f7d2-4c75-b76e-964c5b630cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:15.835169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 734/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:15.873342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16737661-6c1e-430e-9dcf-fcd864087975 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:15.884166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16737661-6c1e-430e-9dcf-fcd864087975 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:15.887710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:15.888338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:15.888618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:15.888890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:15.897630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:15.899116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac51b6-d25f-4d5e-a0bc-27ab852f5f84 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:15.908334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 753/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:18.118593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040b78-33bf-4485-b12e-c1bb19bb25e9 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:18.119196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040b78-33bf-4485-b12e-c1bb19bb25e9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:18.119196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0040b78-33bf-4485-b12e-c1bb19bb25e9 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:18.120568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 735/1488] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:18.125880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724a1fa-addc-49d3-bd1b-badb82b02d77 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:18.126343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724a1fa-addc-49d3-bd1b-badb82b02d77 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:18.126446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724a1fa-addc-49d3-bd1b-badb82b02d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:18.143667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f724a1fa-addc-49d3-bd1b-badb82b02d77 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:18.144835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f724a1fa-addc-49d3-bd1b-badb82b02d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:18.152524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 754/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:18.192317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0040b78-33bf-4485-b12e-c1bb19bb25e9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:18.202736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0040b78-33bf-4485-b12e-c1bb19bb25e9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:18.206190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:18.206836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:18.207117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:18.207384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:18.216023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:18.217435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd77175-0204-4c79-aa13-8fb9a5a7053d service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:18.227454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 736/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:20.484214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bbc6-9148-42bc-b4d6-76b76e119536 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:20.484775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bbc6-9148-42bc-b4d6-76b76e119536 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:20.484775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bbc6-9148-42bc-b4d6-76b76e119536 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:20.486200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 755/1491] 10.222.0.33 () {54 vars in 926 bytes} [Wed Jun 15 00:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:29:20.491492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cbc9-3cba-4e1f-83d2-6fd95346e894 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:20.491904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cbc9-3cba-4e1f-83d2-6fd95346e894 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:20.492045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cbc9-3cba-4e1f-83d2-6fd95346e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:20.510154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-be54cbc9-3cba-4e1f-83d2-6fd95346e894 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:29:20.511355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-be54cbc9-3cba-4e1f-83d2-6fd95346e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:29:20.519013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 737/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Wed Jun 15 00:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:29:20.558471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9bbc6-9148-42bc-b4d6-76b76e119536 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:29:20.568970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9bbc6-9148-42bc-b4d6-76b76e119536 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:29:20.572426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:29:20.573052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:29:20.573328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:29:20.573601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:29:20.582428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:29:20.583873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70efdf6-ca65-48cb-b7fe-2aa0e35e9f84 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:29:20.593968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 756/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:29:30.504754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=33 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:29:30.589128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=33 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:31:02.459162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.459724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.459724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.478391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:31:02.479588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:02.487358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 738/1494] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:31:02.497168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.497577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.497777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.516075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:31:02.517248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:02.525623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 757/1495] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:31:02.529864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.540430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f3d40-520b-4860-88cc-1a0a53af5429 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.546325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.547008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.547294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.547561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.548283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:31:02.548579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.549585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.549585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:31:02.549585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:31:02.550460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.564265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 739/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:31:02.572329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.581940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463760f9-92ad-4f47-a57e-1cbdfe2f2063 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.586873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.587526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.587806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.588072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.588798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.590569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.590569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:31:02.590569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:31:02.591799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.628326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 758/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:31:02.632655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.643268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d25f-95d5-4606-b82b-11e97e3e56a2 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.706814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.706814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.706814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.706814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.707310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.707310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.707310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:31:02.707310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:31:02.707310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.730715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 740/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:31:02.732403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.742955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00350-364e-40e7-9304-fe2730aa0c86 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.747816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.748481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.748786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.749061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.749917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.750972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.750972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:31:02.750972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:31:02.751855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.765948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 759/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:31:02.769942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.780457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65b2a-d0f2-401b-8257-f12acfc74d13 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.785157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.785808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.786114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.786390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.787172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.788822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.788822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:31:02.788822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:31:02.789700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddb94-1290-4a58-baa8-b91fc01d333f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.802619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 741/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:31:02.806807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:02.816702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec27b-b4b7-400a-84fe-6f402920cf6d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:02.821210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:02.822073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.822350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.822619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] PATH_INFO: `/v3/projects/de2702ea7a83408381c438a9daa7cd63/users/11975385edc54e8a87fa6db9b0aa70f6/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.837504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2702ea7a83408381c438a9daa7cd63, user_id=11975385edc54e8a87fa6db9b0aa70f6, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:02.838530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:02.838530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:31:02.838530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:31:02.840495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d28a5-0c56-4844-b4e4-13a181256243 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:02.854674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 760/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:31:02 2022] PUT /identity/v3/projects/de2702ea7a83408381c438a9daa7cd63/users/11975385edc54e8a87fa6db9b0aa70f6/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:31:02.866230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef8f6-96bd-415d-8e23-327451849650 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.866519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef8f6-96bd-415d-8e23-327451849650 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.866816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef8f6-96bd-415d-8e23-327451849650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.893730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ef9ef8f6-96bd-415d-8e23-327451849650 None None] MFA Rules not processed for user `11975385edc54e8a87fa6db9b0aa70f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:31:02.894894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ef9ef8f6-96bd-415d-8e23-327451849650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:02.957283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 742/1502] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:31:02.970794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92ffe-daa5-43c6-bfc8-fc01b4ef7cc9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.971125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92ffe-daa5-43c6-bfc8-fc01b4ef7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.971406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c92ffe-daa5-43c6-bfc8-fc01b4ef7cc9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.972786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 761/1503] 10.222.0.33 () {54 vars in 963 bytes} [Wed Jun 15 00:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:31:02.978824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bbc98-92ee-4a57-a0b5-7d95e38a68ad None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:02.978824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bbc98-92ee-4a57-a0b5-7d95e38a68ad None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:02.978824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bbc98-92ee-4a57-a0b5-7d95e38a68ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:02.996086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-401bbc98-92ee-4a57-a0b5-7d95e38a68ad None None] MFA Rules not processed for user `7fb0a7dcacc745229341c1c02c575721`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:31:02.997267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-401bbc98-92ee-4a57-a0b5-7d95e38a68ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:03.040335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:31:03.048851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 743/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Wed Jun 15 00:31:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:31:03.056490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c92ffe-daa5-43c6-bfc8-fc01b4ef7cc9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:03.067113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c92ffe-daa5-43c6-bfc8-fc01b4ef7cc9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:03.071257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:03.071944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.072221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.072500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.080969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:03.082392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:03.091972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 762/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:31:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:31:03.133449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8def-06e7-4b1a-b476-49f4ace82190 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.133739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8def-06e7-4b1a-b476-49f4ace82190 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.134014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8def-06e7-4b1a-b476-49f4ace82190 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.137911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 744/1506] 10.222.0.33 () {52 vars in 829 bytes} [Wed Jun 15 00:31:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:31:03.138990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:03.201644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7dd6d-8c35-47b7-823f-9d6812eaabe8 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:03.206137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:03.207027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.207309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.207582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] PATH_INFO: `/v3/users/11975385edc54e8a87fa6db9b0aa70f6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.208776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11975385edc54e8a87fa6db9b0aa70f6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:03.210307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:03.210307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:31:03.210307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:31:03.211534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:03.275510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11975385edc54e8a87fa6db9b0aa70f6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.276299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11975385edc54e8a87fa6db9b0aa70f6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.285555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11975385edc54e8a87fa6db9b0aa70f6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.300761 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11975385edc54e8a87fa6db9b0aa70f6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.321732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:31:03.324040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11975385edc54e8a87fa6db9b0aa70f6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.331372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 763/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:31:03 2022] DELETE /identity/v3/users/11975385edc54e8a87fa6db9b0aa70f6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:31:03.336715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.337033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.337314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.354724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:31:03.355961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:03.415191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 745/1508] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:31:03.425302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:03.426728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:03.484348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d92b-b392-429a-a6bc-094f36474112 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:03.487938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:03.488591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.488872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.489148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.498977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:03.500435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf28067-22c1-4b7c-903f-7c92180e9848 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:03.510259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 764/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:31:03.871422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:31:03.872851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:31:03.885289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1311f8c-796c-43d3-8434-1d780cc2e749 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:31:03.890098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:31:03.890993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:31:03.891278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:31:03.891554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] PATH_INFO: `/v3/projects/de2702ea7a83408381c438a9daa7cd63` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:31:03.892767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2702ea7a83408381c438a9daa7cd63)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:31:03.893921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:31:03.893921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:31:03.893921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:31:03.895166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:31:03.957535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2702ea7a83408381c438a9daa7cd63', 'request_id': 'req-f431bf3c-f444-408b-a87b-4e45d569eb32'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.963507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2702ea7a83408381c438a9daa7cd63', 'request_id': 'req-f431bf3c-f444-408b-a87b-4e45d569eb32'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.973783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2702ea7a83408381c438a9daa7cd63', 'request_id': 'req-f431bf3c-f444-408b-a87b-4e45d569eb32'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.993131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] The token cache is being invalidate because project de2702ea7a83408381c438a9daa7cd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:31:03.993350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.994080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f431bf3c-f444-408b-a87b-4e45d569eb32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:31:03.995653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 746/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:31:03 2022] DELETE /identity/v3/projects/de2702ea7a83408381c438a9daa7cd63 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:31:13.505548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:31:13.989418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:32:38.644420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:38.645013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:38.645013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:38.646366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:38.651690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0b35-cf3f-4dd6-a254-6ee445488d58 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:38.652112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0b35-cf3f-4dd6-a254-6ee445488d58 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:38.652233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0b35-cf3f-4dd6-a254-6ee445488d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:38.670359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-914b0b35-cf3f-4dd6-a254-6ee445488d58 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:38.671567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-914b0b35-cf3f-4dd6-a254-6ee445488d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:38.729823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:38.896066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:38.898358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:38.973630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e411f-2e3c-461d-8532-7c3630d5f80d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:38.979486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:38.980128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:38.980410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:38.980686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:38.988764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:38.990368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8df988-36c2-4191-836f-3d78663d3cf7 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:38.999663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 766/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:39.873666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ebfa-9ae4-4213-9ed2-eb28220416cc None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:39.874175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ebfa-9ae4-4213-9ed2-eb28220416cc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:39.874175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ebfa-9ae4-4213-9ed2-eb28220416cc None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:39.875592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:39.880363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3d1e-1b8c-4f03-9ae1-b2e68c25c44a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:39.880782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3d1e-1b8c-4f03-9ae1-b2e68c25c44a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:39.880920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3d1e-1b8c-4f03-9ae1-b2e68c25c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:39.898521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f26e3d1e-1b8c-4f03-9ae1-b2e68c25c44a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:39.899750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f26e3d1e-1b8c-4f03-9ae1-b2e68c25c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:39.907487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:40.060049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2ebfa-9ae4-4213-9ed2-eb28220416cc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:40.074699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2ebfa-9ae4-4213-9ed2-eb28220416cc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:40.080066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:40.080698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:40.080979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:40.081249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:40.089807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:40.091437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa6c9b-eda7-456f-a213-2ab87fec1eff service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:40.101290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 749/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:40.383753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991405-6ea2-46b3-9c07-b9a7663607d9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:40.383950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991405-6ea2-46b3-9c07-b9a7663607d9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:40.384223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc991405-6ea2-46b3-9c07-b9a7663607d9 None None] PATH_INFO: `/v3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:40.385532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 768/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Jun 15 00:32:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:32:40.391603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628330-a679-466d-94df-a25806569e18 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:40.391894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628330-a679-466d-94df-a25806569e18 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:40.392348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628330-a679-466d-94df-a25806569e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:40.413941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a9628330-a679-466d-94df-a25806569e18 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:40.415186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a9628330-a679-466d-94df-a25806569e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:40.478639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 750/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 00:32:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:49.409504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf8b7-3a0e-4f57-b609-b3c7c62d3676 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:49.410072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf8b7-3a0e-4f57-b609-b3c7c62d3676 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:49.410072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf8b7-3a0e-4f57-b609-b3c7c62d3676 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:49.411387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 769/1519] 10.222.0.33 () {54 vars in 960 bytes} [Wed Jun 15 00:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:49.417134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1d08-f397-4b43-a987-851ddccc2635 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:49.417574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1d08-f397-4b43-a987-851ddccc2635 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:49.417685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1d08-f397-4b43-a987-851ddccc2635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:49.435549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b91a1d08-f397-4b43-a987-851ddccc2635 None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:49.436752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b91a1d08-f397-4b43-a987-851ddccc2635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:49.445703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 751/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Wed Jun 15 00:32:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:49.453046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cf8b7-3a0e-4f57-b609-b3c7c62d3676 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:49.466830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cf8b7-3a0e-4f57-b609-b3c7c62d3676 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:49.471749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:49.472377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:49.472660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:49.472936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:49.481909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:49.483606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5f8a-8618-43ba-9061-0c19f5684d13 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:49.493543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 770/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:51.355188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ad1a-c1b6-46c5-9b92-830eba75f468 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:51.355626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ad1a-c1b6-46c5-9b92-830eba75f468 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:51.355732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ad1a-c1b6-46c5-9b92-830eba75f468 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:51.357079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:51.362003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac369d-21fe-4883-b97f-26fff4678cf1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:51.362428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac369d-21fe-4883-b97f-26fff4678cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:51.362579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac369d-21fe-4883-b97f-26fff4678cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:51.382875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3bac369d-21fe-4883-b97f-26fff4678cf1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:51.384080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3bac369d-21fe-4883-b97f-26fff4678cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:51.391921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:51.545630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ad1a-c1b6-46c5-9b92-830eba75f468 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:51.559950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ad1a-c1b6-46c5-9b92-830eba75f468 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:51.565301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:51.565927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:51.566206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:51.566480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:51.574486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:51.576126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8154-8485-4a3b-ba96-96c32ddc448b service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:51.585905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 753/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:52.449458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6eed3-3629-43cb-a794-01473ef71181 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:52.450036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6eed3-3629-43cb-a794-01473ef71181 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:52.450036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6eed3-3629-43cb-a794-01473ef71181 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:52.451449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 00:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:52.456317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e398-c8d6-4597-98ac-0dfde7a63384 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:52.456750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e398-c8d6-4597-98ac-0dfde7a63384 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:52.456869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e398-c8d6-4597-98ac-0dfde7a63384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:52.474548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ea54e398-c8d6-4597-98ac-0dfde7a63384 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:52.476624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ea54e398-c8d6-4597-98ac-0dfde7a63384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:52.484480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 00:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:32:52.635856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6eed3-3629-43cb-a794-01473ef71181 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:52.649875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6eed3-3629-43cb-a794-01473ef71181 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:52.655178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:52.655800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:52.656077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:52.656347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:52.664945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:52.666527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12085a8c-a8e8-48ec-b5e9-cdc4d8acb6e7 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:52.675916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 773/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:56.875826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:56.876362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:56.876362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:56.877638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:56.882885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024fe8-e1d6-4a3c-b197-97a132c10ff7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:56.883324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024fe8-e1d6-4a3c-b197-97a132c10ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:56.883454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024fe8-e1d6-4a3c-b197-97a132c10ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:56.901596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c0024fe8-e1d6-4a3c-b197-97a132c10ff7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:56.902799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c0024fe8-e1d6-4a3c-b197-97a132c10ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:56.910506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:32:56.946392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:56.948079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:57.004282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1280f35-2caf-4e78-9919-a704320efe12 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:57.008292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:57.008929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:57.009207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:57.009478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:57.018310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:57.019774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7a81-0e5a-4d70-b11f-a1c4892c1ec7 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:57.025195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:32:57.030186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 756/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:32:59.152792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dbe-0989-4ffc-bbf2-e477326fa8ae None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:59.153403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dbe-0989-4ffc-bbf2-e477326fa8ae None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:59.153403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41dbe-0989-4ffc-bbf2-e477326fa8ae None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:59.154713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:32:59.159794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f1b1-c478-40d7-bb1c-99b235acae20 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:59.160216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f1b1-c478-40d7-bb1c-99b235acae20 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:59.160334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f1b1-c478-40d7-bb1c-99b235acae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:59.178453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b195f1b1-c478-40d7-bb1c-99b235acae20 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:32:59.179690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b195f1b1-c478-40d7-bb1c-99b235acae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:32:59.187638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:32:59.222949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41dbe-0989-4ffc-bbf2-e477326fa8ae None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:32:59.234431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41dbe-0989-4ffc-bbf2-e477326fa8ae None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:32:59.237947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:32:59.238573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:32:59.238887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:32:59.239161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:32:59.248620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:32:59.250112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2715a-eca9-4b3c-9632-4bc0cc1686ab service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:32:59.259276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 776/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:33:02.771871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:02.772546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:02.772546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:02.773900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:33:02.779138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea2054-bf69-4df9-ab19-c67c4081e114 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:02.779566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea2054-bf69-4df9-ab19-c67c4081e114 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:02.779676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea2054-bf69-4df9-ab19-c67c4081e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:02.797227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0dea2054-bf69-4df9-ab19-c67c4081e114 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:33:02.798417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0dea2054-bf69-4df9-ab19-c67c4081e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:33:02.806713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:33:03.071003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:33:03.072654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:33:03.128957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7675-c8a0-493d-abbe-1ad1bef7559f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:33:03.132643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:33:03.133281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:03.133565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:03.133841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:03.142794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:33:03.144223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84230bc0-e473-4ee3-82e4-381add26271a service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:33:03.153481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 759/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:33:05.310577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:05.311191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:05.311191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:05.312537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 00:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:33:05.317715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f26-41df-446e-9988-ec1f74df04d6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:05.318128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f26-41df-446e-9988-ec1f74df04d6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:05.318286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f26-41df-446e-9988-ec1f74df04d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:05.335998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-71d50f26-41df-446e-9988-ec1f74df04d6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:33:05.337189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-71d50f26-41df-446e-9988-ec1f74df04d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:33:05.344955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 00:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:33:05.609754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:33:05.611480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:33:05.618995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=65 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:33:05.625395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973761f-c8ec-4d60-bbc9-1d19a18b9837 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:33:05.628935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:33:05.629567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:33:05.629847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:33:05.630117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:33:05.639021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:33:05.640432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a57c5-85de-4540-acce-eec80bd2d5d3 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:33:05.650183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 779/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:33:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:33:15.330985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=32 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:33:15.645636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:32.957908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b7a0-78a0-4b73-8ead-24f8498c49db None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:32.959205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b7a0-78a0-4b73-8ead-24f8498c49db None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:32.960329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b7a0-78a0-4b73-8ead-24f8498c49db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:32.972637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2296d-a080-4f37-aaf6-da37f6e6527a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:32.973644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2296d-a080-4f37-aaf6-da37f6e6527a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:32.974139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2296d-a080-4f37-aaf6-da37f6e6527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:32.984662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1121b7a0-78a0-4b73-8ead-24f8498c49db None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:32.985868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1121b7a0-78a0-4b73-8ead-24f8498c49db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:32.996765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8bc2296d-a080-4f37-aaf6-da37f6e6527a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:32.998283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8bc2296d-a080-4f37-aaf6-da37f6e6527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.004169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 761/1540] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.005458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a867-f90f-4976-9050-5788cef28942 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.005739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a867-f90f-4976-9050-5788cef28942 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.006019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a867-f90f-4976-9050-5788cef28942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.008809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 780/1541] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:33.019710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd941b-b878-4fea-8c27-a3779b951b44 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.020282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd941b-b878-4fea-8c27-a3779b951b44 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.020738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd941b-b878-4fea-8c27-a3779b951b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.040754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-92dd941b-b878-4fea-8c27-a3779b951b44 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.042254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-92dd941b-b878-4fea-8c27-a3779b951b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.043625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-faa9a867-f90f-4976-9050-5788cef28942 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.044973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-faa9a867-f90f-4976-9050-5788cef28942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.051919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 781/1542] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.053688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.054139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.054583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.059860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 762/1543] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.061282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3057-f4da-4c29-b51d-01f4ed538d00 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.061552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3057-f4da-4c29-b51d-01f4ed538d00 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.061826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3057-f4da-4c29-b51d-01f4ed538d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.079468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.080959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.090105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 782/1544] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.091219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.099876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8aba3057-f4da-4c29-b51d-01f4ed538d00 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.101384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8aba3057-f4da-4c29-b51d-01f4ed538d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.102325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514350b-650c-4a8c-8f09-e662ff6605ce None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.107417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.108465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.109013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.109555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.110455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.110964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.112131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:34:33.112131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:34:33.112131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:34:33.113354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.120183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 763/1545] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.121580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.121729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.121988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.127634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 783/1546] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.128654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.143117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502241d-1386-49af-8002-38d368b41a66 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.152463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.153414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.153869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.154007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.154420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.155346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.155485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.155937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.157981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f0b84-e26a-4ec8-9e96-7998e13e9e7d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.172244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 784/1547] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.173937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.174231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.174507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.177944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 764/1548] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.179220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.193754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.195678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.197420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed342f-6dfc-4ceb-a597-57f9d5945c43 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.204261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 785/1549] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.204910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.212092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.213110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.213607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.214117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.215077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.216394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd7c4d-4352-404c-a9d9-5687fe7bb1ed None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.216714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:34:33.216714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:34:33.216714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:34:33.218312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.221123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.221778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.222065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.222343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.223224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.223538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.225171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.238652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 786/1550] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.238966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.249769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d490c1-c55d-42f0-86b0-0a68c1b41e75 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.254851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.255552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.256074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.256414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.257958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.260722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.275025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 765/1551] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.276261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.309287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 787/1552] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.310214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.387260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b786478-82e6-4179-9224-b4d4b96ee84a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.417912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f113-3d9a-4f01-8cdb-9a51af90d72e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.423104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.423772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.424057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.424386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.425090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.427769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.464872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 788/1553] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.465687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.468471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.469280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.469675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.470045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.471011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.471413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.473172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.492224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 766/1554] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.493177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.504024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02f78b-ecd8-4a8e-87b6-9f14c0752318 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.533092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:33.536955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051d74-1b11-4477-a763-98b70c392cc5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.542718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.543442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.543717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.543985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.544696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.545683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:34:33.545683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:34:33.545683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:34:33.546922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f74c91-a3dd-44bb-b8be-398f7a7846f4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.552655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:33.563735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.564408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.564690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.564965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.565654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.567601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eccaa-4d2d-4b05-8afc-fb9d19bb9572 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.570453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 789/1555] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.571842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba49ed3-4660-4c78-a6b8-b92f924d6835 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.572283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba49ed3-4660-4c78-a6b8-b92f924d6835 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.572560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba49ed3-4660-4c78-a6b8-b92f924d6835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.588772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 767/1556] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.591875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.591875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.591875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.592116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-eba49ed3-4660-4c78-a6b8-b92f924d6835 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.592519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eba49ed3-4660-4c78-a6b8-b92f924d6835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.600361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 790/1557] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.609166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109255d-1712-4e0f-9cec-4c47c4cd69cf None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.609166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109255d-1712-4e0f-9cec-4c47c4cd69cf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.609166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109255d-1712-4e0f-9cec-4c47c4cd69cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.609435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.610413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.618857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 768/1558] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.620668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.626686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d109255d-1712-4e0f-9cec-4c47c4cd69cf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.627912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d109255d-1712-4e0f-9cec-4c47c4cd69cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.630216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cffda3-3db9-470a-aa92-8e21844f7f39 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.636298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.636643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 791/1559] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.636944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.637223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.637495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.637839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.638120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.638234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.638388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.640098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.656999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.658288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.664589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 769/1560] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.664589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.669965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 792/1561] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.670759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.674176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f502b-1e3f-4ef3-abfe-c48e9c222b4e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.682701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.683586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.683720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7531ee-3941-4efe-af11-d37d98aa6b8a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.683884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.684157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.684961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.687120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3cf05-105a-4a60-b551-e19edff1ff32 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.688286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.688956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.689235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.689506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.690229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.691854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.703233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 793/1562] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.704636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.714432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3625ce8-9182-47c9-9202-33688c3ea57d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.715663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 770/1563] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.716865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a0f-3cbc-4faa-a202-865a76f33fd3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.717153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a0f-3cbc-4faa-a202-865a76f33fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.717458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a0f-3cbc-4faa-a202-865a76f33fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.735162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-91f39a0f-3cbc-4faa-a202-865a76f33fd3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.736352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-91f39a0f-3cbc-4faa-a202-865a76f33fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.775150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.775816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.776098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.776374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.777092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.778696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13db3b-dd58-45ee-b4c9-1913595b8b3e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.788963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 794/1564] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.790153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.790438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.790714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.799008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 771/1565] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.800475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c61f-4744-471a-9097-6f3be591d350 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.800757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c61f-4744-471a-9097-6f3be591d350 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.801029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c61f-4744-471a-9097-6f3be591d350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.811886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.813079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.818766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f130c61f-4744-471a-9097-6f3be591d350 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.820057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f130c61f-4744-471a-9097-6f3be591d350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.820998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 795/1566] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.824755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.828136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 772/1567] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.829347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.829631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.829904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.830796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad1ad-cb1d-4d6e-ad8d-4d2b651d2470 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.835642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.836276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.836556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.836824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.837543 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.839156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.848227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:33.849106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 796/1568] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.849419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:33.849966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.865471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 773/1569] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:33.867000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.871988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9322-2019-4a20-b299-c6afb29b568e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.876291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce0ac9-cf62-4739-b2fd-055ec89ec8cd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.876556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.877209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.877492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.877766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.878481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.880142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.880826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.881510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.881793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.882355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.882757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.884684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.893188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 797/1570] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.893188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.900313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab787a0-8286-49ad-a975-388f88715538 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.904596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 774/1571] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:33 2022] POST /identity/v3/users => generated 287 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:33.904923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.905465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.905581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.905830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.906152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.906845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.908434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.916015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288ec9-36c1-410a-8460-e835646a40c0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.919471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 798/1572] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.920246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.920608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.921248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.921533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.921812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.922940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.923252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.924833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.931033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af54ba-3cdf-4de8-8c16-b17903dda6c9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.936271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.936900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.937182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.937460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.937606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 775/1573] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.938175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.938370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.938470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.940096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.953134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 799/1574] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.953910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.954389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4baec2-2fea-4199-a956-dea95cd51a46 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.959000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.960236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.960539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.960910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.961641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.961948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.963609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.965927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ca19d-5c13-4d2c-aa33-3847cd37d14a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.970491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.971158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.971445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.971713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.972425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.973995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.976195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 776/1575] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:33.977053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.984447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 800/1576] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:33.985265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:33.987445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f0eb-0a6e-4711-9eed-346e67d95ef1 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.992748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:33.993384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:33.993669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:33.993949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:33.994671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:33.995002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:33.995170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2780-cd9d-4f74-97a6-2abc7c45c58f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:33.996591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:33.999671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.000549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.000829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.001103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] PATH_INFO: `/v3/projects/42c353894cf64e6ca9e2e4311632eef4/users/7ba112db5e4547f5bae76ab64e5cb23c/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.008926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 777/1577] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:34.009794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.015959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c353894cf64e6ca9e2e4311632eef4, user_id=7ba112db5e4547f5bae76ab64e5cb23c, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.018821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.019305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfe6e-ec2f-46e5-95bf-3049c056ddad admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.023931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.024648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.024879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.025157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.025877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.027493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.032150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 801/1578] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:33 2022] PUT /identity/v3/projects/42c353894cf64e6ca9e2e4311632eef4/users/7ba112db5e4547f5bae76ab64e5cb23c/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:34.032999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.043441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 778/1579] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.043441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.051867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b19b-a19d-41e7-91f7-9ca968cf208e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.059835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.064154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.098070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167d20-fd50-4f9d-803f-c4413859c578 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.103070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.103952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.104234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.104509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] PATH_INFO: `/v3/projects/ad57875727e24965a37faca9c9a30984/users/db8e18e9735c4cd0afb79921383e8b67/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.112013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.112665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.112945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.113222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.113915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.116403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.119250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad57875727e24965a37faca9c9a30984, user_id=db8e18e9735c4cd0afb79921383e8b67, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.121928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.137419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 802/1580] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:34 2022] PUT /identity/v3/projects/ad57875727e24965a37faca9c9a30984/users/db8e18e9735c4cd0afb79921383e8b67/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:34.138200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.141699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 779/1581] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.142511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.202986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407672c-7e5a-464c-a4c4-2050f29f51c0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.203550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148bd4d-29a7-4a9f-94b2-0836a51a69b4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.207726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.208145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.208384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.208664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.208775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.208940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.209038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.209307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.209658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.210012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.211651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.211920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.237710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 803/1582] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.238505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.243585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 780/1583] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.244451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.308218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d9988-3548-415f-b40c-416c50a2ba9f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.313447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.314452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.314871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.315004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63865f-3c50-4c37-a51e-c90d655c8a16 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.315316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] PATH_INFO: `/v3/projects/828186402a37485ab1b716d3f1d60787/users/d618ff755fd34edc92b395365fec482c/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.320117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.320768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.321071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.321330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.322146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.324101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.330099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=828186402a37485ab1b716d3f1d60787, user_id=d618ff755fd34edc92b395365fec482c, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.332777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fe688-c092-407d-a1dc-66c662e370fc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.346316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 804/1584] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:34 2022] PUT /identity/v3/projects/828186402a37485ab1b716d3f1d60787/users/d618ff755fd34edc92b395365fec482c/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:34.347769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e299a5-d59c-4d23-890d-f7a0ee7e5d1e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.348078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e299a5-d59c-4d23-890d-f7a0ee7e5d1e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.348493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e299a5-d59c-4d23-890d-f7a0ee7e5d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.357752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 781/1585] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.359048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.371933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-69e299a5-d59c-4d23-890d-f7a0ee7e5d1e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:34.373115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69e299a5-d59c-4d23-890d-f7a0ee7e5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:34.373588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.383529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.426015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31f194-eff6-42f3-8189-3c12f953a187 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.431440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.432184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.432535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.432878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.433710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.434352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 805/1586] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:34.435472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.435635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.435921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.436195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.451184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 782/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.451184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.455615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:34.456804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:34.460651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dab40-afe3-43e9-bc15-9364f5574c2c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.465320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.465482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 806/1588] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:34.466210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.466472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.466616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.466983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.467660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.469567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.480208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdae1c-0563-4a7b-a290-e814916850a2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.485496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.486165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.486449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.486722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.487442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.492769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d998221-4b1f-4b41-b638-f6c871849013 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.492937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 783/1589] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.495075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.505722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d00b0-23b4-4d1e-84f7-bd87a970fd41 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.510833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.511128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 807/1590] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.513619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.513619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.513619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.514628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.514989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.515215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.515908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.517828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.541073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 784/1591] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.541073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.541379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] MFA Rules not processed for user `d618ff755fd34edc92b395365fec482c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:34.542630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:34.548505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3190b1-09a4-4b37-958c-376a4e690e80 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.553066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.553738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.554014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.554288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.555000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.556894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.577860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 785/1592] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/users => generated 296 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:34.578941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.609061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 808/1593] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:34.609803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.620518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18383371-ab1c-41c8-8d9e-565b1db2ab48 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.625443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.626227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.626499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.626945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] PATH_INFO: `/v3/projects/6973acb409cc4a92b0e724716ae4c05e/users/7cd9c3cbaf43400cba3eff7794dd19c7/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.639458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3842b12-861f-4293-befd-d25e20b15435 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.643554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.644358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.644624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.644899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.648134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6973acb409cc4a92b0e724716ae4c05e, user_id=7cd9c3cbaf43400cba3eff7794dd19c7, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.651291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.657566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.659183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.664516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 809/1594] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:34 2022] PUT /identity/v3/projects/6973acb409cc4a92b0e724716ae4c05e/users/7cd9c3cbaf43400cba3eff7794dd19c7/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:34.665467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.671625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 786/1595] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:34.672517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.681952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12426b7-836e-4f72-85d0-8bd283bb9689 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.730509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027380-e3f9-4cff-93c4-e6bb09b9f2a5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.734576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.735320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.735665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.736133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.745648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.747436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:34.754420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.755261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.755830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.756364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.757729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.759950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.775132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 787/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.775132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.784135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf86835-920a-481f-aad1-64a8aa20cf07 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.789159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.789878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.790123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.790402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.791423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.792931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.804045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 788/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.804875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.811164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.812742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.817140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c9f46-96e1-48ba-9f88-e1d709bdba3d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.822953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.823624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.824022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 810/1598] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:34.824164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.824164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.824969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.825100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.826496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487cd3e-6cb0-44de-a070-cf1354f4dd60 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.834949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f53a3-7237-4165-98b6-706a158e1875 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.837963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 789/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.839490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81058-c72e-4236-a5c2-42da0141abd9 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.839784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81058-c72e-4236-a5c2-42da0141abd9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.840066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81058-c72e-4236-a5c2-42da0141abd9 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.841841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 790/1600] 10.222.0.33 () {54 vars in 961 bytes} [Wed Jun 15 00:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:34.842969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686ed87-d105-461e-8643-b6788c94231e None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.843257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686ed87-d105-461e-8643-b6788c94231e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.843537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686ed87-d105-461e-8643-b6788c94231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.844693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.845645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.846165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.846606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.847585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.850213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.861498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 811/1601] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.861498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.875461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f28d-e324-4a3c-b719-875c034f9caf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.883002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.883724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.884010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.884391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.884795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a686ed87-d105-461e-8643-b6788c94231e None None] MFA Rules not processed for user `7cd9c3cbaf43400cba3eff7794dd19c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:34.885180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.886539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a686ed87-d105-461e-8643-b6788c94231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:34.887175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.906266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 812/1602] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.906266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.920043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1656b35-0067-456c-95cc-20dd49e660b5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.926159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.927095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.927251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.927455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.929032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.931167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.942817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 813/1603] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.944550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.956481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168057e-3a1f-4fed-9b35-c4de0317aebd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:34.961677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:34.962377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.962726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.962972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.963710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:34.966279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:34.980800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 791/1604] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:34.982470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:34.982888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:34.983260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:34.983609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 814/1605] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:34.984544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:34.995879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a620b-aaeb-4f60-8e03-6d0118235d6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.001956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.002966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.003358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.003826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.004826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.007935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.008099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.009368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.024496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 815/1606] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:35.026003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.040321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888812c4-e349-4cda-aa53-196718035e6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.046186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.047224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.047565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.047850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] PATH_INFO: `/v3/projects/32c13d8e3a764cf59f81e17eaca1c853/users/eceb9f042cb24343892f09454c7e685a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.069870 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c13d8e3a764cf59f81e17eaca1c853, user_id=eceb9f042cb24343892f09454c7e685a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.072690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.091200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 816/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:35 2022] PUT /identity/v3/projects/32c13d8e3a764cf59f81e17eaca1c853/users/eceb9f042cb24343892f09454c7e685a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:35.094676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.098387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 792/1608] 10.222.0.33 () {58 vars in 1058 bytes} [Wed Jun 15 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.099622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.114765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6e67c-b638-4d76-a5fb-96848642c44a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.189927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8371b-bfdd-41e0-b640-0371288b0c52 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.196294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:35.200308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.211523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.212537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.213506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] PATH_INFO: `/v3/projects/47d48d22515e469fa53498b17ed80be7/users/dc56e0e37b3d4445a69391142935abcf/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.222036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.222981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.223270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.223550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] PATH_INFO: `/v3/projects/2d3c2ce190cc4882a6daef0dd44bb88c/users/631ea7a4c5314020b766de2fc7e7974d/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.237713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d48d22515e469fa53498b17ed80be7, user_id=dc56e0e37b3d4445a69391142935abcf, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.242111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2203e-9b49-4441-b1a8-d2d7ca703965 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.246000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d3c2ce190cc4882a6daef0dd44bb88c, user_id=631ea7a4c5314020b766de2fc7e7974d, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.247305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:34:35.247305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:34:35.247305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:34:35.250013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664dc32-98a1-4d41-84a2-7a67ab8596f9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.266870 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 817/1609] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:35 2022] PUT /identity/v3/projects/47d48d22515e469fa53498b17ed80be7/users/dc56e0e37b3d4445a69391142935abcf/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:35.270604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411c44-1eba-441a-aab9-865fae4a8c71 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.270752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411c44-1eba-441a-aab9-865fae4a8c71 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.272382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411c44-1eba-441a-aab9-865fae4a8c71 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.275792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 818/1610] 10.222.0.33 () {54 vars in 961 bytes} [Wed Jun 15 00:34:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:35.276950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9411c44-1eba-441a-aab9-865fae4a8c71 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.280464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 793/1611] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:35 2022] PUT /identity/v3/projects/2d3c2ce190cc4882a6daef0dd44bb88c/users/631ea7a4c5314020b766de2fc7e7974d/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:35.282097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.283578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.284052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.299274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:35.305856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.307863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.372834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9411c44-1eba-441a-aab9-865fae4a8c71 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.378616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.379714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.380000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.380320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] PATH_INFO: `/v3/projects/c41af6aa819743d380018ec1eed0a793/users/b1430d9ff64e4a9fa2f40bb31a098090/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.399453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 794/1612] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.399669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41af6aa819743d380018ec1eed0a793, user_id=b1430d9ff64e4a9fa2f40bb31a098090, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.400899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.403733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2ddce-fbc1-4554-9a17-c3e8271a6e61 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.429038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 819/1613] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:35 2022] PUT /identity/v3/projects/c41af6aa819743d380018ec1eed0a793/users/b1430d9ff64e4a9fa2f40bb31a098090/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:35.430499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d01f2b-444f-42e6-abb9-801db5474865 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.431208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d01f2b-444f-42e6-abb9-801db5474865 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.431208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d01f2b-444f-42e6-abb9-801db5474865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.453560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d1d01f2b-444f-42e6-abb9-801db5474865 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.455285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d1d01f2b-444f-42e6-abb9-801db5474865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.512585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c0c5e-df61-4541-9d98-5b68fa396841 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.546192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 820/1614] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.548314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95b7-5e41-45cc-badd-8145e36fa03e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.548703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95b7-5e41-45cc-badd-8145e36fa03e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.549151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95b7-5e41-45cc-badd-8145e36fa03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.573891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-42ca95b7-5e41-45cc-badd-8145e36fa03e None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.575744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-42ca95b7-5e41-45cc-badd-8145e36fa03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.622266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.623875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.624320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.624927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.642820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:35.672551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 821/1615] 10.222.0.33 () {58 vars in 1058 bytes} [Wed Jun 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.674420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8e5f-d534-439b-9624-e7d18f23e6a1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.674832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8e5f-d534-439b-9624-e7d18f23e6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.675213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe8e5f-d534-439b-9624-e7d18f23e6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.699194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6efe8e5f-d534-439b-9624-e7d18f23e6a1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.700679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6efe8e5f-d534-439b-9624-e7d18f23e6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.711201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 822/1616] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.715912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.716296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.716795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.720491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.722193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.736188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:35.737367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:35.747120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 795/1617] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:35.747383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 823/1618] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:35.748263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.748577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.769801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:35.825657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17d7a-e2d5-4868-afd3-fb325336f0f6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.830458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.831412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.832027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.832466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.848745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.851896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.875611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 824/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:35.877806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.902640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0767-6f00-408e-a36a-c09e36f12b29 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.918702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4acbb-0e7b-4766-b19a-d9c4979c54a4 service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.919069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.921319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.921604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.921884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.923828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.925484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.925484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.925801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:35.942323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:35.944942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:35.961485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 796/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:35.963003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:35.978672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846e731-26f7-4e3d-9800-e322b43cf8bf service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:35.984662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:35.985537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:35.985757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:35.986447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.006106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.008751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.029033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 797/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:36.030007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:36.041139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9f87-382b-46aa-8f6e-e95f1cd16efc service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:36.045594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:36.046655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.046989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.047488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.050331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.051877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a64593-4e66-4082-ad3f-cece6e44f7f7 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.060386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.062520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067094d5-bfb0-4fc2-840a-31c3f4d8ac79 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.077824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 825/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:36.079173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85acb-346e-47d5-b1c5-11652e6a5fec None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.079468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85acb-346e-47d5-b1c5-11652e6a5fec None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.082367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85acb-346e-47d5-b1c5-11652e6a5fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.104083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 798/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:36.121237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d398e-7d85-41bf-818b-4db77749c58a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.121531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d398e-7d85-41bf-818b-4db77749c58a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.121810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d398e-7d85-41bf-818b-4db77749c58a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.123883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 799/1624] 10.222.0.33 () {54 vars in 912 bytes} [Wed Jun 15 00:34:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:34:36.130659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.131127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.132071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.138904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a7b85acb-346e-47d5-b1c5-11652e6a5fec None None] MFA Rules not processed for user `7ba112db5e4547f5bae76ab64e5cb23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:36.139822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a7b85acb-346e-47d5-b1c5-11652e6a5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:36.155885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:36.157656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:36.186339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 800/1625] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:36.199575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:36.301897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e085f-65de-497c-a760-01dec4652eb4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:36.307207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:36.308069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.308235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.308457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.321650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.323278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.336443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 826/1626] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:36.337363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 801/1627] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:34:36.341750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f468c4-ca78-4c8a-85f6-de216df77ebc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.342069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f468c4-ca78-4c8a-85f6-de216df77ebc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.342353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f468c4-ca78-4c8a-85f6-de216df77ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.374910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f7f468c4-ca78-4c8a-85f6-de216df77ebc None None] MFA Rules not processed for user `7ba112db5e4547f5bae76ab64e5cb23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:36.377786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7f468c4-ca78-4c8a-85f6-de216df77ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:36.398801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 827/1628] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:36.517832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:36.528926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f85ec-6dbc-4fdf-b30a-eef5cc010eb6 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.529120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f85ec-6dbc-4fdf-b30a-eef5cc010eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.529359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f85ec-6dbc-4fdf-b30a-eef5cc010eb6 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.531007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7ca07-73b6-4bb6-bd0b-bc3bc951c00e service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:36.531147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 828/1629] 10.222.0.33 () {54 vars in 912 bytes} [Wed Jun 15 00:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:34:36.536078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.536396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.536677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.538566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:36.539352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.539763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.540049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.549458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.551885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edaa45-803c-4282-b1b0-f40e01d0cd2a service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.557568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:36.558967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:36.586463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 829/1630] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:36.598232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 802/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:36.604291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:36.615276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d25d6-2f26-45d1-8f2d-45b84693e43d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:36.627766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:36.628504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:36.628791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:36.629072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:36.653465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:36.657898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabad9-dd45-4721-9133-0984c742d022 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:36.680243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 830/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:37.519164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:37.519164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:37.519164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:37.519164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 803/1633] 10.222.0.33 () {54 vars in 912 bytes} [Wed Jun 15 00:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:37.520260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:37.520260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:37.520260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:37.598985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:37.616608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:37.617847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:37.634039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 831/1634] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:34:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:37.669837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:37.677862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:37.702523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53337a62-a151-4612-b670-a21ab997fa58 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:37.716589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:37.717252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:37.717532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:37.717801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:37.744637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:37.746363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fe7cf-a5a3-43ae-b17b-86464413f62e service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:37.755328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:37.771271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 804/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:37.982586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:38.000024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cad9-851d-40cc-99c9-d55a16f9fa2d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:38.008397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:38.009196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:38.010083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:38.010380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] PATH_INFO: `/v3/users/d618ff755fd34edc92b395365fec482c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:38.015955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d618ff755fd34edc92b395365fec482c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:38.021104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:38.105498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd618ff755fd34edc92b395365fec482c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:38.106452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd618ff755fd34edc92b395365fec482c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:38.124872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd618ff755fd34edc92b395365fec482c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:38.136874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd618ff755fd34edc92b395365fec482c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:38.158269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd618ff755fd34edc92b395365fec482c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:38.179716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70de3-561c-4d27-8945-eb06802a93f3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:38.180300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70de3-561c-4d27-8945-eb06802a93f3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:38.181276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c70de3-561c-4d27-8945-eb06802a93f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:38.184095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 832/1636] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:34:37 2022] DELETE /identity/v3/users/d618ff755fd34edc92b395365fec482c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:38.213166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-52c70de3-561c-4d27-8945-eb06802a93f3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:38.214385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-52c70de3-561c-4d27-8945-eb06802a93f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:38.323331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 805/1637] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:38.347401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:38.351063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:38.427620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a482a-4529-41b4-80cb-16414bbe271f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:38.432798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:38.433470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:38.433803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:38.434083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:38.446709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:38.448365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:38.464132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 833/1638] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:38.901890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:38.901890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:38.902567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:38.991291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] MFA Rules not processed for user `631ea7a4c5314020b766de2fc7e7974d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:38.994004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:38.998754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.000749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:39.195437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 806/1639] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:39.196552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.198048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:39.210548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3079-860e-4374-8e5d-78119cb75f18 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.228514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.229188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.229470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.229746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.230276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5abbf-d7b1-470e-a417-ce7cd4117a04 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.232068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:39.237773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.238587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.242178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.245215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.247104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.252236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.275671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.277313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.290315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 834/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:34:39.327094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 807/1641] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:39 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:39.328468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.338365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:39.348867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd6ca-87e4-47f0-bc03-96cb77530629 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.511574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.512264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.512544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.512818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.514517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.516507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.577417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 835/1642] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:39 2022] POST /identity/v3/users => generated 309 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:39.581306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.601954 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ca55-4287-45ea-af00-51dd42927b50 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.612873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.613567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.613846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.614116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.614903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.616771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.651949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.657490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 808/1643] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:39.665638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ae1e-3e58-4a24-adef-462536934e24 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.683632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.684570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.684849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.685121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] PATH_INFO: `/v3/projects/2eef39097e0e4e9b9dc3b52270753eac/users/f35a4ecb02b349a8b3e765e77445ab47/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.715864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eef39097e0e4e9b9dc3b52270753eac, user_id=f35a4ecb02b349a8b3e765e77445ab47, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.719138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.751766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 836/1644] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:39 2022] PUT /identity/v3/projects/2eef39097e0e4e9b9dc3b52270753eac/users/f35a4ecb02b349a8b3e765e77445ab47/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:34:39.757096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.771704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307277a-a1a1-4584-b7d2-910340a5c1f8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.918161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:39.937012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.937782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.938134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.938548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:39.939392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:39.941192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:39.962503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 809/1645] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:39.966184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:39.981620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76056d31-17a4-42f0-b2f3-b067ffbd8874 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:39.995691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:39.996800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:39.997192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:39.997732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] PATH_INFO: `/v3/projects/2eef39097e0e4e9b9dc3b52270753eac/users/f35a4ecb02b349a8b3e765e77445ab47/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.002374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eef39097e0e4e9b9dc3b52270753eac, user_id=f35a4ecb02b349a8b3e765e77445ab47, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:40.005202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03136e1-7a57-4cd5-8cb8-aa160930cbd2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:40.016616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:40.020022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:40.035340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 837/1646] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:39 2022] PUT /identity/v3/projects/2eef39097e0e4e9b9dc3b52270753eac/users/f35a4ecb02b349a8b3e765e77445ab47/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:40.047126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed5ca9-5ea9-4fc8-a33f-c714a800e77c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:40.197520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:40.200654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.200950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.204285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] PATH_INFO: `/v3/projects/828186402a37485ab1b716d3f1d60787` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.211012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828186402a37485ab1b716d3f1d60787)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:40.217132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:40.351835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828186402a37485ab1b716d3f1d60787', 'request_id': 'req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.362103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828186402a37485ab1b716d3f1d60787', 'request_id': 'req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.379471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828186402a37485ab1b716d3f1d60787', 'request_id': 'req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.399086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] The token cache is being invalidate because project 828186402a37485ab1b716d3f1d60787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:34:40.399482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.400461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c648f820-fcc1-4692-a2e1-a62b9b4ef8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.406248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 810/1647] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:34:40 2022] DELETE /identity/v3/projects/828186402a37485ab1b716d3f1d60787 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:40.407530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.407936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.408325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.439407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:40.455162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:40.575923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 838/1648] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:40.583903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.584858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.585328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.645882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:40.645882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:40.658320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:40.660823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:40.695894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 811/1649] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:40.697856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f745e-a052-402e-be46-bc4b98d1be13 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:40.699241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:40.719384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:40.720664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.722388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.723049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] PATH_INFO: `/v3/users/7cd9c3cbaf43400cba3eff7794dd19c7` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.731703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd9c3cbaf43400cba3eff7794dd19c7)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:40.732868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50badedc-48a7-44b1-a177-e756a8b431a3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:40.739961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:40.757030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:40.762525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.763547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.763958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.767698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:40.768784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:40.774549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:40.812724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 812/1650] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:34:40.817497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:40.837618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad867d63-cc5f-44bf-814b-81fb07b40bab admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:40.852620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:40.853576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.854002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.855256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:40.856099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:40.862638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:40.878218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd9c3cbaf43400cba3eff7794dd19c7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.879437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd9c3cbaf43400cba3eff7794dd19c7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.887171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:40.899853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd9c3cbaf43400cba3eff7794dd19c7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.918959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd9c3cbaf43400cba3eff7794dd19c7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.930148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 813/1651] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:40 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:40.934250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:40.956440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a1afa9a0-db27-4434-9dc8-4e94cbaeb9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd9c3cbaf43400cba3eff7794dd19c7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:40.970171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 839/1652] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:34:40 2022] DELETE /identity/v3/users/7cd9c3cbaf43400cba3eff7794dd19c7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:40.977253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:40.978356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:40.980041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.025941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:41.029546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:41.054356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:41.094923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eed0d-f590-48cf-b900-1c5fdacd0dea admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:41.143619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 840/1653] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:41.176257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:41.179179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:41.254411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:41.255278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.255690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.258289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.259739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:41.265580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:41.318412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb3b4-3b18-4db6-8a26-e8dcb5ae70ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:41.319725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 814/1654] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:40 2022] POST /identity/v3/users => generated 283 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:41.327769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:41.329728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:41.330585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:41.331301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.332960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.335752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.343937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3c709-5cdd-4d01-a115-5e6805ac567b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:41.359194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:41.364777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:41.365094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:41.366494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.367411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.367958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.368790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:41.370561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:41.385320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 815/1655] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:41.389199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:41.397829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 841/1656] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:34:41.403499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25407c8-1a85-4c98-ab9a-23cbaab3c68d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:41.409484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:41.410303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.410710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.411083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.412926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:41.415011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a9507-12ba-4830-b285-4e8df3c1ccd3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:41.431912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 816/1657] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:41.435780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:41.450714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb01c6-a34e-4a63-94c3-22a33b9b1ecd service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:41.457333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:41.460729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.461352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.461709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] PATH_INFO: `/v3/projects/fadc6d1ee79441ec8cdd586d0e0f6176/users/6199592d4dfe48c4b9468a2380e2d1cf/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.510665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fadc6d1ee79441ec8cdd586d0e0f6176, user_id=6199592d4dfe48c4b9468a2380e2d1cf, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:41.519591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:41.561500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 842/1658] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:41 2022] PUT /identity/v3/projects/fadc6d1ee79441ec8cdd586d0e0f6176/users/6199592d4dfe48c4b9468a2380e2d1cf/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:34:41.569638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:41.570015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:41.570361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:41.646291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] MFA Rules not processed for user `6199592d4dfe48c4b9468a2380e2d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:41.651106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:42.197803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 817/1659] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 633 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:42.202241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.284959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:42.335811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301018-d78f-47f3-9092-54fab6595223 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.351911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.356300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.356300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.357807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.360121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.365302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:42.432985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 843/1660] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:42 2022] POST /identity/v3/projects => generated 365 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:42.433091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.458109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559c545-09be-4739-b670-afd0a9624cb1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.551910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:42.599732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.601083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.601547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.602018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.602924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.605104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:42.639230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 818/1661] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:42 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:42.645104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.661030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f704b-afc4-439e-9480-4770f158abcd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.670039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.670931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.681340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.681703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.682549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.696965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:42.708793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 844/1662] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:42.716872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.732086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbf121-cb2f-428e-8236-b8c553c5f542 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.748976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.749645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.749922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.750204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] PATH_INFO: `/v3/projects/0629476d34ac43a7a480438c96b657a2/users/78b91d71f4ec48e082773cafd0071a6e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.786125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0629476d34ac43a7a480438c96b657a2, user_id=78b91d71f4ec48e082773cafd0071a6e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.790612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:42.814419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 819/1663] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:42 2022] PUT /identity/v3/projects/0629476d34ac43a7a480438c96b657a2/users/78b91d71f4ec48e082773cafd0071a6e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:42.818745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.833525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0611ef-e9c5-49c7-a60a-b264f9b441ad admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.932301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.933646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.934137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.934616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.935886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.937730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fe51a-1dd6-44f5-8933-3eae4a0316e8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:42.956376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 845/1664] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:42.956693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:42.971884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a060e1-a6d1-4613-a768-30b6b0969653 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:42.985134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:42.986289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:42.986712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:42.987219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] PATH_INFO: `/v3/projects/0629476d34ac43a7a480438c96b657a2/users/78b91d71f4ec48e082773cafd0071a6e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:42.991594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0629476d34ac43a7a480438c96b657a2, user_id=78b91d71f4ec48e082773cafd0071a6e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:42.994736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd414e-06cb-4e07-9013-5fd8fe72f20e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:43.011928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 820/1665] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:42 2022] PUT /identity/v3/projects/0629476d34ac43a7a480438c96b657a2/users/78b91d71f4ec48e082773cafd0071a6e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:43.027588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:43.027861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:43.027986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:43.064788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] MFA Rules not processed for user `78b91d71f4ec48e082773cafd0071a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:43.067098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.180947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:43.181262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:43.181600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:43.211535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:43.213112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.239505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 846/1666] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:43.247275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:43.248963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.299509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 821/1667] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:43.360076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:43.369333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:43.373024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.389554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e396e-4de3-4325-b4bb-5d8c219af933 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:43.407252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:43.407942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:43.408225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:43.408500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:43.431145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:43.435079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b0863-6ca1-464b-aa33-f97b1b1ad271 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:43.471421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 847/1668] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:34:43.532019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:43.533178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd97c-8145-4eba-a851-69af5de10ee0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:43.535361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:43.537127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:43.555460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:43.556888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:43.557484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:43.557484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] PATH_INFO: `/v3/projects/6973acb409cc4a92b0e724716ae4c05e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:43.566248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6973acb409cc4a92b0e724716ae4c05e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:43.570969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:43.618331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] MFA Rules not processed for user `f35a4ecb02b349a8b3e765e77445ab47`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:43.619674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.694941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:43.702576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6973acb409cc4a92b0e724716ae4c05e', 'request_id': 'req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:43.773150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6973acb409cc4a92b0e724716ae4c05e', 'request_id': 'req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:43.792222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6973acb409cc4a92b0e724716ae4c05e', 'request_id': 'req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:43.803397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 848/1669] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:43.804225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:43.813069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.831891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] The token cache is being invalidate because project 6973acb409cc4a92b0e724716ae4c05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:34:43.832513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:43.843927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:43.846821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 822/1670] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:34:43 2022] DELETE /identity/v3/projects/6973acb409cc4a92b0e724716ae4c05e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:43.848264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:43.853509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:43.978592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cda8e-8e0e-4968-83e9-20dbdb645fa6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:44.022577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ffd3-b7b8-4c74-b41b-7056a24e5ff7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:44.027536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:44.028838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.029276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.029641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.035335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:44.109679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:44.110511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.110806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.111108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.116040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:44.148940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:44.154401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:44.156822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a81052-3821-42ee-818b-f779c72d6dc5 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:44.170239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 823/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:44.171874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.172305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.172695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.220309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:44.222449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:44.278268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:44.280661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e326-834d-49fb-8f06-e98bc75814af service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:44.306255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 849/1672] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:34:44.432242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 824/1673] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:44.439586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.439873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.440153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.480818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:44.502596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:44.539818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 850/1674] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:44.540014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:44.551582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6489-c134-41ee-a729-9994be0c315c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:44.601245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:44.612923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.619263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.619859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.620858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:34:44.622597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:44.634341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:44.717335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:44.720834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 825/1675] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:34:44.732104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecaf16-a18f-4af6-a0de-73b79d51c7d4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:44.750799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:44.752647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:44.752937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:44.753219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:44.753939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:44.767453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:44.938239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 851/1676] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:44.948226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:44.990157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf75d1-fd55-4848-a412-d8d7c6301fd9 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:45.044993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:45.275228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:45.276797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:45.277281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:45.277814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:45.282530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:45.287373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:45.359728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 826/1677] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:44 2022] POST /identity/v3/users => generated 287 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:45.365915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:45.384034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a387954-d3a8-4f94-9f28-169c4f1ef61d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:45.409713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:45.410423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:45.410711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:45.411121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:45.411885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:45.419287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:45.444951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 852/1678] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:45.448247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:45.463220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e575-b87a-4c21-8f91-a785643a61d9 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:45.483916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:45.484620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:45.484905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:45.485181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:45.486070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:45.491389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6db5e-c0bd-42ef-8744-f08c26069de3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:45.519757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 827/1679] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:45.523920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:45.541043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dc196-59d6-48d7-82fa-541bee921517 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:45.566914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:45.574472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:45.574797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:45.575039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] PATH_INFO: `/v3/projects/58ef2d58b11443718ffc22e7977b6e57/users/124672bb407b41ea99db0bd611f5865c/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:45.643794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ef2d58b11443718ffc22e7977b6e57, user_id=124672bb407b41ea99db0bd611f5865c, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:45.649886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:45.685719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 853/1680] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:45 2022] PUT /identity/v3/projects/58ef2d58b11443718ffc22e7977b6e57/users/124672bb407b41ea99db0bd611f5865c/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:45.705249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7873-ee57-48e1-a12d-ad21d675c79c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:45.705559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7873-ee57-48e1-a12d-ad21d675c79c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:45.705823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7873-ee57-48e1-a12d-ad21d675c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:45.802542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4e1f7873-ee57-48e1-a12d-ad21d675c79c None None] MFA Rules not processed for user `124672bb407b41ea99db0bd611f5865c`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:45.804527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4e1f7873-ee57-48e1-a12d-ad21d675c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:46.083707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 828/1681] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:46.084387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:46.259705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.259952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.260182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:46.314352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda264e-5f26-497d-87cc-5b0495660db9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:46.344478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:46.345295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.347507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.347967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:46.348836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:46.352666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:46.357649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:46.363422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:46.436154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 854/1682] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:34:46 2022] POST /identity/v3/projects => generated 371 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:34:46.441422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:46.589798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 829/1683] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:46.609110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:46.646124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:46.647640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:46.671952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694cb6-57f6-4b73-8e35-c4a6a574a312 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:46.711056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d57dcc-ed36-4ea6-9b8e-ce3c9b4aca81 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:46.740014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:46.741433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.741820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.742763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:46.749445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:46.751224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:46.793011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:46.824634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 855/1684] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:34:46 2022] POST /identity/v3/users => generated 292 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:34:46.825833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:46.856562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:46.857237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.857669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.857949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:46.858356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587bbb1-6485-47a6-8245-bd960b3016c9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:46.870804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:46.871724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.872244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.872523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:46.873268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:46.874982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:46.882659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:46.884175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:46.923916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 830/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:34:46.931233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 856/1686] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:34:46.933924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:46.964453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33187-0e73-497e-924b-e8b52d09d25c service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:46.977733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:46.981010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:46.983277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:46.984124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] PATH_INFO: `/v3/projects/077e8909fbef424cb254b7718d375136/users/941bb0b8d3b84a0d9639c8c73aab8230/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:47.055918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077e8909fbef424cb254b7718d375136, user_id=941bb0b8d3b84a0d9639c8c73aab8230, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:47.070558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:47.115185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 831/1687] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:46 2022] PUT /identity/v3/projects/077e8909fbef424cb254b7718d375136/users/941bb0b8d3b84a0d9639c8c73aab8230/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:47.116737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:47.159860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794ecf1-7a07-43ff-b3c3-a5d2fca91ccf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:47.329917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:47.330619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:47.330928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:47.331203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:47.331938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:47.333921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da5bae-9e3c-4647-8342-accf7cbbf979 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:47.359131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 857/1688] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:34:47.364001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:47.392249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f0f30-2fc8-4f17-9baf-653ef1c1c384 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:47.419770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:47.423303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:47.423690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:47.423975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] PATH_INFO: `/v3/projects/077e8909fbef424cb254b7718d375136/users/941bb0b8d3b84a0d9639c8c73aab8230/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:47.437055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] RBAC: Authorizing `identity:create_grant(project_id=077e8909fbef424cb254b7718d375136, user_id=941bb0b8d3b84a0d9639c8c73aab8230, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:47.464036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:47.510223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 832/1689] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:34:47 2022] PUT /identity/v3/projects/077e8909fbef424cb254b7718d375136/users/941bb0b8d3b84a0d9639c8c73aab8230/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:47.520784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff270-75cb-414f-9984-62461456017a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:47.521300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff270-75cb-414f-9984-62461456017a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:47.521394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff270-75cb-414f-9984-62461456017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:47.589224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-16cff270-75cb-414f-9984-62461456017a None None] MFA Rules not processed for user `941bb0b8d3b84a0d9639c8c73aab8230`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:47.590477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-16cff270-75cb-414f-9984-62461456017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:47.768737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 858/1690] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:47.781201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:47.783138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:47.912045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9c082-0eaa-4118-bdf6-9dce55ac318f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:47.932863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:47.933923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:47.935013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:47.935498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:47.965460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:47.974155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e363a-ca00-409f-9b27-bb7f1c28d65e service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:48.003371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 833/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:48.131379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cff270-75cb-414f-9984-62461456017a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:48.148308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cff270-75cb-414f-9984-62461456017a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:48.153429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:48.154599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:48.155553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:48.156110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:48.242868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:48.330132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:48.332542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c65772-043d-4fd9-a19b-3607c6abdcff service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:48.357199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 859/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:51.549991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed957-f7c3-441e-8a23-98a12410635b None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:51.550605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed957-f7c3-441e-8a23-98a12410635b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:51.550750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed957-f7c3-441e-8a23-98a12410635b None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:51.553337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 834/1693] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:51.852845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369e0b-b342-4def-8262-dacaa8c2122e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:51.853371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369e0b-b342-4def-8262-dacaa8c2122e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:51.853586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369e0b-b342-4def-8262-dacaa8c2122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:51.902458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-35369e0b-b342-4def-8262-dacaa8c2122e None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:51.914013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-35369e0b-b342-4def-8262-dacaa8c2122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:51.967028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:51.967313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:51.967592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.048297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] MFA Rules not processed for user `db8e18e9735c4cd0afb79921383e8b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:52.049665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:52.129606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 860/1694] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:52.270727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646377c-f674-479a-b645-0bc9a40a17ef None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.271365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646377c-f674-479a-b645-0bc9a40a17ef None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.271850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646377c-f674-479a-b645-0bc9a40a17ef None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.278917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 861/1695] 10.222.0.33 () {54 vars in 960 bytes} [Wed Jun 15 00:34:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:52.295007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.295313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.295852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.328260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 835/1696] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:52.352818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:52.354369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:52.364745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:52.367360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:52.407235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 862/1697] 10.222.0.33 () {58 vars in 1057 bytes} [Wed Jun 15 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:34:52.409474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:52.441021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a681cc5-0b35-478b-8705-48b151443ede None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:52.454471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:52.456023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.463861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.464770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.470540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:52.491989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:52.496063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214854-03d6-4534-b411-fb5fae3f9a03 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:52.538677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 863/1698] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:52.541342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034f7ea-b0b6-41f3-a1f6-ee6e351d495f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.541637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034f7ea-b0b6-41f3-a1f6-ee6e351d495f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.541917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034f7ea-b0b6-41f3-a1f6-ee6e351d495f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.569811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 864/1699] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:34:52 2022] GET /identity => generated 270 bytes in 30 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:52.647712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a347d18-7ad0-47e7-9a2b-db75f08c5af0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:52.659287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:52.662192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.663724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.667008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.700086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:52.702039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:52.744853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 836/1700] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:52.850741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b44c-d7c0-45ff-8cfc-74ad77921a28 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:52.851051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b44c-d7c0-45ff-8cfc-74ad77921a28 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:52.851330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b44c-d7c0-45ff-8cfc-74ad77921a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:52.909207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-46e0b44c-d7c0-45ff-8cfc-74ad77921a28 None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:52.911078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-46e0b44c-d7c0-45ff-8cfc-74ad77921a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:52.936942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 865/1701] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:53.294498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:53.320611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9af48-aa69-4499-8d8a-5875cd61f53b service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:53.351719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:53.352384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:53.358442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:53.358737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:53.405251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:53.406816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20107-d6b2-4de9-9f94-941057a9671e service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:53.439283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 837/1702] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:56.379107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:56.379627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:56.380982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:56.446043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] MFA Rules not processed for user `b1430d9ff64e4a9fa2f40bb31a098090`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:56.461514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:56.582301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 866/1703] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:56.594040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c112b3-2581-48ae-9fa4-a7c2bf3e4b9c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:56.594897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c112b3-2581-48ae-9fa4-a7c2bf3e4b9c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:56.594897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c112b3-2581-48ae-9fa4-a7c2bf3e4b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:56.618619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-98c112b3-2581-48ae-9fa4-a7c2bf3e4b9c None None] MFA Rules not processed for user `b1430d9ff64e4a9fa2f40bb31a098090`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:56.620308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-98c112b3-2581-48ae-9fa4-a7c2bf3e4b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:56.629305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 838/1704] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:56.642318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:56.644031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:56.668277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de530d-dfa9-4b59-a590-158be0b1986e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:56.681799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:56.685958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:56.687128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:56.687547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:56.703631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:56.706996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916c3ef-a067-4d21-991e-96eccda07e44 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:56.725703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 867/1705] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:57.035964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285fe4-f065-462b-8e60-c6f778c2980a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:57.036258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285fe4-f065-462b-8e60-c6f778c2980a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:57.036605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285fe4-f065-462b-8e60-c6f778c2980a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:57.038978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 839/1706] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:34:57.065957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:57.066436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:57.066828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:57.155409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] MFA Rules not processed for user `07f71bb8eebf4ec4999192359dafb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:57.156864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:57.178865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 868/1707] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:57.229626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285fe4-f065-462b-8e60-c6f778c2980a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:57.258279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285fe4-f065-462b-8e60-c6f778c2980a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:57.274961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:57.278348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:57.278788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:57.280057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:57.309789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:57.321373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:57.349128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 840/1708] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:34:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:34:57.402417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:57.404060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:57.451910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:34:57.608007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c29b5d-0ef9-4bb5-a546-43ddecbed30b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:57.632110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:57.634980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:57.636969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:57.637380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] PATH_INFO: `/v3/users/631ea7a4c5314020b766de2fc7e7974d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:57.640097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631ea7a4c5314020b766de2fc7e7974d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:57.645175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:57.783496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631ea7a4c5314020b766de2fc7e7974d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:57.786656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631ea7a4c5314020b766de2fc7e7974d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:57.805507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631ea7a4c5314020b766de2fc7e7974d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:57.820438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631ea7a4c5314020b766de2fc7e7974d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:57.852053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ea5d054-1e0d-4b91-9784-21afcf101610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631ea7a4c5314020b766de2fc7e7974d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:57.874626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 869/1709] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:34:57 2022] DELETE /identity/v3/users/631ea7a4c5314020b766de2fc7e7974d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:59.397887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:59.434389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f752e-dd50-42a5-8211-dc83d4d6b737 service placement] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:34:59.460623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07910c-05a0-42e3-948e-576859997dee None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:59.461133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07910c-05a0-42e3-948e-576859997dee None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:59.461205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07910c-05a0-42e3-948e-576859997dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:59.498113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa07910c-05a0-42e3-948e-576859997dee None None] MFA Rules not processed for user `b1430d9ff64e4a9fa2f40bb31a098090`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:34:59.499409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa07910c-05a0-42e3-948e-576859997dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:59.661035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 870/1710] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:34:59.678206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:34:59.679629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:34:59.680567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:34:59.682761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] PATH_INFO: `/v3/projects/2d3c2ce190cc4882a6daef0dd44bb88c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:34:59.686392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d3c2ce190cc4882a6daef0dd44bb88c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:34:59.690704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:34:59.848212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d3c2ce190cc4882a6daef0dd44bb88c', 'request_id': 'req-dc4b0dbe-d56b-44a8-ae37-7462e9937409'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:59.883925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d3c2ce190cc4882a6daef0dd44bb88c', 'request_id': 'req-dc4b0dbe-d56b-44a8-ae37-7462e9937409'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:59.930559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d3c2ce190cc4882a6daef0dd44bb88c', 'request_id': 'req-dc4b0dbe-d56b-44a8-ae37-7462e9937409'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:59.963293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] The token cache is being invalidate because project 2d3c2ce190cc4882a6daef0dd44bb88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:34:59.963619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:59.966485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dc4b0dbe-d56b-44a8-ae37-7462e9937409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:34:59.975122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 841/1711] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:34:59 2022] DELETE /identity/v3/projects/2d3c2ce190cc4882a6daef0dd44bb88c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:34:59.976321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07910c-05a0-42e3-948e-576859997dee None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:34:59.977869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa07910c-05a0-42e3-948e-576859997dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:34:59.999237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07910c-05a0-42e3-948e-576859997dee None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:00.052747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:00.115658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:00.116561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:00.116840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:00.117115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] PATH_INFO: `/v3/users/f35a4ecb02b349a8b3e765e77445ab47` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:00.123107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35a4ecb02b349a8b3e765e77445ab47)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:00.125698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:00.283792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35a4ecb02b349a8b3e765e77445ab47'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:00.288839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35a4ecb02b349a8b3e765e77445ab47'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:00.327184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35a4ecb02b349a8b3e765e77445ab47'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:00.356619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35a4ecb02b349a8b3e765e77445ab47'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:00.401340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9c8c1c9a-cdeb-4eb1-ba89-0e0ee95bba04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35a4ecb02b349a8b3e765e77445ab47'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:00.420696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 871/1712] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:34:59 2022] DELETE /identity/v3/users/f35a4ecb02b349a8b3e765e77445ab47 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:00.788920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:00.789390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:00.789473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:00.790900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 842/1713] 10.222.0.33 () {54 vars in 912 bytes} [Wed Jun 15 00:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 00:35:00.812786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:00.813274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:00.813548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:00.857305 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:00.870261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:01.031912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 872/1714] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:35:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:01.439225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:01.441166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:01.485136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:01.591177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:01.593899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:01.620871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217204b-3182-4dd7-9a64-915606d2fcd4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:01.627979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:01.629081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:01.629592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:01.630203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:01.644338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:01.653959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23c0d-dee0-4d10-8ff2-0772671684d3 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:01.665780 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c169-59aa-4cfc-a125-ba8f8c1fe498 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:01.681759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 873/1715] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:01.692707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:01.696820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:01.697384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:01.697861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] PATH_INFO: `/v3/users/6199592d4dfe48c4b9468a2380e2d1cf` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:01.699880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6199592d4dfe48c4b9468a2380e2d1cf)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:01.701774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: WARNING py.warnings [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:35:01.701774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: warnings.warn(msg) Jun 15 00:35:01.701774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: Jun 15 00:35:01.704380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:01.863544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6199592d4dfe48c4b9468a2380e2d1cf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:01.866016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6199592d4dfe48c4b9468a2380e2d1cf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:01.867661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6199592d4dfe48c4b9468a2380e2d1cf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:01.872368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:01.872856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:01.872952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:01.895418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6199592d4dfe48c4b9468a2380e2d1cf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:01.924891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7c761383-848a-493a-a9cc-e9e36ba68a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6199592d4dfe48c4b9468a2380e2d1cf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:01.951244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] MFA Rules not processed for user `eceb9f042cb24343892f09454c7e685a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:01.953018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:01.987483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 843/1716] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:01 2022] DELETE /identity/v3/users/6199592d4dfe48c4b9468a2380e2d1cf => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:01.994667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:01.995204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:01.995605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.027942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:02.030003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.075519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:02.136777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 844/1717] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:02.137661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 874/1718] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:02.166242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:02.168411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:02.169440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.170181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.233491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:02.269904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542967b-1089-4406-a16a-8ef330226e76 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:02.275777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:02.275777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:02.276279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:02.276647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.290249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:02.293019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:02.306026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 845/1719] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:02.328551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e37f-6ed9-4863-bff0-06fcdd7d700b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:02.572884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:02.573534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:02.573959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:02.574245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.600869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:02.605518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.648987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ac1bc-26d7-4925-b9cc-96dc7291d12d service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:02.659235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:02.660352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:02.660771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:02.661233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] PATH_INFO: `/v3/projects/2eef39097e0e4e9b9dc3b52270753eac` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.663407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eef39097e0e4e9b9dc3b52270753eac)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:02.666399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:02.789383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:02.799043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eef39097e0e4e9b9dc3b52270753eac', 'request_id': 'req-19c28a74-9e1b-41dd-9314-bd003cf09307'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:02.799226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63a9d-c354-4889-8558-e8b7db722ea5 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:02.808340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eef39097e0e4e9b9dc3b52270753eac', 'request_id': 'req-19c28a74-9e1b-41dd-9314-bd003cf09307'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:02.821374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eef39097e0e4e9b9dc3b52270753eac', 'request_id': 'req-19c28a74-9e1b-41dd-9314-bd003cf09307'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:02.840264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] The token cache is being invalidate because project 2eef39097e0e4e9b9dc3b52270753eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:02.840734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:02.841529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-19c28a74-9e1b-41dd-9314-bd003cf09307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:02.843615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 846/1720] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:02 2022] DELETE /identity/v3/projects/2eef39097e0e4e9b9dc3b52270753eac => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:02.844885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 875/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:02.845479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494929b-db7b-4cbf-895e-e57de99a1cd1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:02.845858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494929b-db7b-4cbf-895e-e57de99a1cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:02.846242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494929b-db7b-4cbf-895e-e57de99a1cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.848444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:02.850046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:02.851740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:02.874321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6494929b-db7b-4cbf-895e-e57de99a1cd1 None None] MFA Rules not processed for user `db8e18e9735c4cd0afb79921383e8b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:02.876095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6494929b-db7b-4cbf-895e-e57de99a1cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.947426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:02.953679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:02.955035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 847/1722] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:02.984391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:03.138441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 876/1723] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:03.146144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:03.147050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:03.147594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:03.192426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:03.196296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:03.226184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 848/1724] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:03.231136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:03.251587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5b88-9b05-46f2-b92b-fbbab88106e7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:03.283497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:03.288701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:03.291562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:03.291955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:03.295146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:03.297598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:03.306588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:03.356682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 877/1725] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:03.359878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:03.376428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b9a3f-a0ba-45b5-bf95-2fa8e0f4ad34 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:03.405008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:03.408977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:03.408977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:03.413697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:03.414914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:03.422935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:03.551815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 849/1726] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:03 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:03.554495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:03.572508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cefb-31ea-4aa4-9ecd-bc436bdc360b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:04.122196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:04.122896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.123109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.123384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.128324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:04.133276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:04.177926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 878/1727] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:03 2022] POST /identity/v3/users => generated 282 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:04.181943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:04.200002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5eb6-b161-4dcc-a89d-757df715716b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:04.217660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:04.219233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.220243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.221557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.224895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:04.228912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:04.260433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 850/1728] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:04.265290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:04.278579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf4da3-d544-4dee-a1ea-b496a98742ed admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:04.285144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:04.286387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.287308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.287844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.288898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:04.292312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9dd25-0d21-454d-a6ff-497cb5edac18 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:04.311058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 879/1729] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:04.315940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:04.339926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc344c-1ee4-40db-bc4d-dff9ed6329cf admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:04.361233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:04.382879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.383149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.383440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] PATH_INFO: `/v3/projects/87b820411401415186a49a29b0099e23/users/7ce638d4294c4855a0bdafd37a56171c/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.446676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b820411401415186a49a29b0099e23, user_id=7ce638d4294c4855a0bdafd37a56171c, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:04.450802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b685653-1de9-4ef7-b31b-4a04656918e4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:04.466288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.466579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.467050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.523019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:04.556163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 851/1730] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:04 2022] PUT /identity/v3/projects/87b820411401415186a49a29b0099e23/users/7ce638d4294c4855a0bdafd37a56171c/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:04.558485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.558485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:04.559258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:04.566210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] MFA Rules not processed for user `dc56e0e37b3d4445a69391142935abcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:04.569045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:04.634345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] MFA Rules not processed for user `7ce638d4294c4855a0bdafd37a56171c`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:04.637921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:04.785268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 852/1731] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:04.793941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:04.796512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:04.852713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 880/1732] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:04.853748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:04.921675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:04.986873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad75e1-0678-4c4e-9fd9-b608f738da22 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:04.997636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:04.998544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:04.999725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.000012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] PATH_INFO: `/v3/projects/fadc6d1ee79441ec8cdd586d0e0f6176` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.017047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fadc6d1ee79441ec8cdd586d0e0f6176)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.026650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.114825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308f03-00bd-4472-8eff-ed3c397f1f4d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:05.124312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:05.125700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:05.126128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.126545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.127450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.129438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.189857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 881/1733] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:04 2022] POST /identity/v3/projects => generated 365 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:05.191027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:05.221944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fadc6d1ee79441ec8cdd586d0e0f6176', 'request_id': 'req-7032a476-fc79-47e1-baae-1d7503d3fe6a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.242861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fadc6d1ee79441ec8cdd586d0e0f6176', 'request_id': 'req-7032a476-fc79-47e1-baae-1d7503d3fe6a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.276085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fadc6d1ee79441ec8cdd586d0e0f6176', 'request_id': 'req-7032a476-fc79-47e1-baae-1d7503d3fe6a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.316748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] The token cache is being invalidate because project fadc6d1ee79441ec8cdd586d0e0f6176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:05.316961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.318222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.328783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 853/1734] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:04 2022] DELETE /identity/v3/projects/fadc6d1ee79441ec8cdd586d0e0f6176 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:05.332868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:05.335140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:05.385276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:05.392363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf221a-033b-4759-ac6f-3c7892f922dd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:05.406450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:05.407174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:05.407455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.407732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.408659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.410643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.455662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 882/1735] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:05 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:05.459716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:05.464693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:05.499801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d62c3-274f-4c9f-8384-e10e1f34e1fc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:05.519608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:05.520404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:05.522666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.522666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.523125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.524777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.546892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a476-fc79-47e1-baae-1d7503d3fe6a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:05.557190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 883/1736] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:05.557697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:05.558649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:05.559003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.559330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] PATH_INFO: `/v3/users/78b91d71f4ec48e082773cafd0071a6e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.560821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:05.563796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b91d71f4ec48e082773cafd0071a6e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.566472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.583162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697453a-05e6-4b7f-8455-8292794b3c94 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:05.597361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:05.598265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:05.598546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:05.603886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] PATH_INFO: `/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f/users/1547e6e60aa5435cbc5f4b21ee4adef0/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:05.659613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4371ac3ac245ebb9fa47c19ecf7a0f, user_id=1547e6e60aa5435cbc5f4b21ee4adef0, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:05.662631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:05.673684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b91d71f4ec48e082773cafd0071a6e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.674593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b91d71f4ec48e082773cafd0071a6e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.675365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b91d71f4ec48e082773cafd0071a6e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.695866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b91d71f4ec48e082773cafd0071a6e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.708903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b91d71f4ec48e082773cafd0071a6e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:05.710645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 884/1737] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:05 2022] PUT /identity/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f/users/1547e6e60aa5435cbc5f4b21ee4adef0/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:05.714824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:05.719178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:05.771732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 854/1738] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:05 2022] DELETE /identity/v3/users/78b91d71f4ec48e082773cafd0071a6e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:05.943707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:05.978209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c14ef-9d6b-440f-b694-e375f307010a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:06.157991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:06.161049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.170697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.171350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.173056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:06.183538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:06.207082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 885/1739] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:06.210401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:06.211888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:06.220842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:06.248452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693afc-1040-4465-a2a1-6bbe752742ee admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:06.259530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:06.260461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.260763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.261040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] PATH_INFO: `/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f/users/1547e6e60aa5435cbc5f4b21ee4adef0/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.270732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4371ac3ac245ebb9fa47c19ecf7a0f, user_id=1547e6e60aa5435cbc5f4b21ee4adef0, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:06.274312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21982885-8507-44f3-8a0c-348d4d9b7927 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:06.334248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 886/1740] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:06 2022] PUT /identity/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f/users/1547e6e60aa5435cbc5f4b21ee4adef0/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:06.339353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.339703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.339987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.382916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6bb8b-b9d7-4008-84c4-7cdde3c30d14 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:06.419461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] MFA Rules not processed for user `1547e6e60aa5435cbc5f4b21ee4adef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:06.420706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:06.466548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:06.548509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:06.549329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.549710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.550098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.554100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:06.589280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 887/1741] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:06.606919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:06.608865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:06.728908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:06.731348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:06.758716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 855/1742] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:06.760113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321acd28-6ffd-49df-b278-8d248543be5c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:06.770560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:06.772278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.773901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.774243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.799498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:06.800959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:06.823777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 888/1743] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:06.962390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:06.986448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de9ca7-eca5-4ea4-b27a-970b0881a5f0 service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:06.992408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:06.993079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:06.993360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:06.993636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:06.995812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:07.087990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:07.090288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018309-c553-489d-b699-b10944d233f5 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:07.120248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 856/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:07.761429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:07.763360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:07.788256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:07.955994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcd1af-a9cb-4874-b30f-ab8dae588cdc service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:07.969341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:07.970401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:07.971850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:07.974215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] PATH_INFO: `/v3/projects/0629476d34ac43a7a480438c96b657a2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:07.978138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0629476d34ac43a7a480438c96b657a2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:07.980410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 00:35:07.980410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Jun 15 00:35:07.980410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: Jun 15 00:35:07.984062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:08.126489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0629476d34ac43a7a480438c96b657a2', 'request_id': 'req-0f78a330-34af-4717-afc6-92a518f82d78'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:08.141147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0629476d34ac43a7a480438c96b657a2', 'request_id': 'req-0f78a330-34af-4717-afc6-92a518f82d78'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:08.157472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0629476d34ac43a7a480438c96b657a2', 'request_id': 'req-0f78a330-34af-4717-afc6-92a518f82d78'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:08.204266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] The token cache is being invalidate because project 0629476d34ac43a7a480438c96b657a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:08.205008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:08.205452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0f78a330-34af-4717-afc6-92a518f82d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:08.210593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 889/1745] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:07 2022] DELETE /identity/v3/projects/0629476d34ac43a7a480438c96b657a2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:08.213070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:08.213567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:08.213652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:08.240681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:08.242262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:08.346811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 857/1746] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:08.353152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:08.354265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:08.354265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:08.403855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:08.410159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:08.439410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 890/1747] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:08.440751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:08.456782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e082cd-6d21-45b3-91e8-b353ffbe3525 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:08.467352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:08.469788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:08.470153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:08.472020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:08.472819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:08.474875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:08.483117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:08.524087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:08.539543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 858/1748] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:08.540450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:08.566277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aced89-8959-4695-9dc7-a9ebef207229 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:08.609427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:08.610223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:08.613301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:08.614409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:08.617233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:08.626229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:08.710993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 891/1749] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:08 2022] POST /identity/v3/projects => generated 391 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:08.714191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:08.740776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050584b7-cd4d-475c-a14e-8c032c2255e9 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:08.903500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:08.905456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:08.905812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:08.906159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:08.907184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:08.911273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:08.970854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 859/1750] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:08 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:08.976451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:08.997064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cec52-f673-48d2-af2e-767e32a30a19 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:09.010997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:09.011803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.012319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.012776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.014472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:09.018685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:09.045610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 892/1751] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:09.046938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:09.074812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62e0fd-594a-4c78-b6f6-d8064297a377 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:09.100067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:09.102964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.103243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.103526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.109916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:09.113983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc8bc2-3209-4154-9c3e-d24e3d3e0ad4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:09.195936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:09.207865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 860/1752] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:09.212406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a405770-090d-4e8d-9f23-93bdcc2a33ca admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:09.242899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:09.244212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.244670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.245117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] PATH_INFO: `/v3/projects/5cde0a93d33f4b06ac523adaff83f412/users/933d48390d03440d9d566c39c134ff21/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.295594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cde0a93d33f4b06ac523adaff83f412, user_id=933d48390d03440d9d566c39c134ff21, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:09.302373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:09.376429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 893/1753] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:09 2022] PUT /identity/v3/projects/5cde0a93d33f4b06ac523adaff83f412/users/933d48390d03440d9d566c39c134ff21/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:09.388164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4c97-6eba-4230-af0a-54ba9b28d2b0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.388742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4c97-6eba-4230-af0a-54ba9b28d2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.388953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4c97-6eba-4230-af0a-54ba9b28d2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.487842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b1fd4c97-6eba-4230-af0a-54ba9b28d2b0 None None] MFA Rules not processed for user `933d48390d03440d9d566c39c134ff21`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:09.497671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b1fd4c97-6eba-4230-af0a-54ba9b28d2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:09.709953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 861/1754] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:09.713413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:09.826002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:09.889455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ed57-f36b-49d2-9c9e-dda5956f0df3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:09.898832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8560-2995-4906-a9ac-7e319dff8356 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.899118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8560-2995-4906-a9ac-7e319dff8356 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.899427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8560-2995-4906-a9ac-7e319dff8356 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.901420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 862/1755] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:35:09.902185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:09.902950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.903296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.903593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:09.904546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:09.906411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8b29a-d9e3-4b39-9ef1-7fd21cfed8bd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:09.948209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 894/1756] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:09 2022] POST /identity/v3/projects => generated 393 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:09.952306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b8560-2995-4906-a9ac-7e319dff8356 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:09.967292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:09.967867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:09.967867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.029660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:10.034302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:10.123139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b8560-2995-4906-a9ac-7e319dff8356 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.140953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.141661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.141946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.142232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.144044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:10.145900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:10.207025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:10.245273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 863/1757] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:09 2022] POST /identity/v3/users => generated 303 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:10.246887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.282276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409d1e-a930-4329-8db7-95fa538c555c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.291185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.293189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.293998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.294809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.296685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:10.304596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:10.315123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 895/1758] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:35:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:10.325990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 864/1759] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:10.331985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.347716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995683c-8c06-49dc-a2d2-dee2ac9ea39b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.362805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.363728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.364005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.364280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] PATH_INFO: `/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d/users/1e00da4652cd431fb03c55f9174111d3/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.391767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d636b2a7779e4f12a3e1bc8c666e342d, user_id=1e00da4652cd431fb03c55f9174111d3, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:10.395340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:10.425407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 896/1760] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:10 2022] PUT /identity/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d/users/1e00da4652cd431fb03c55f9174111d3/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:10.431123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.452475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466933c0-6393-4e72-a85e-be57afa9c54e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.584893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.591904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.592193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.592469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.593368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:10.595610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd49371-69f1-4a96-a06b-1d80936944dc admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:10.614590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 865/1761] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:10.615813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.643487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc256f84-f128-48c1-bab7-bed734584884 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.653265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.654442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.654746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.655098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] PATH_INFO: `/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d/users/1e00da4652cd431fb03c55f9174111d3/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.657277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d636b2a7779e4f12a3e1bc8c666e342d, user_id=1e00da4652cd431fb03c55f9174111d3, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:10.661211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:10.672756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 897/1762] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:10 2022] PUT /identity/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d/users/1e00da4652cd431fb03c55f9174111d3/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:10.683538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e608-7350-4b40-9ed7-5ec8a154da60 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.683860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e608-7350-4b40-9ed7-5ec8a154da60 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.684408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e608-7350-4b40-9ed7-5ec8a154da60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.732951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ad78e608-7350-4b40-9ed7-5ec8a154da60 None None] MFA Rules not processed for user `1e00da4652cd431fb03c55f9174111d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:10.734437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ad78e608-7350-4b40-9ed7-5ec8a154da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:10.760303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.762171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:10.932590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 866/1763] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:10.932590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.932590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.932590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:10.933281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 867/1764] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:35:10.939959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:10.941879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:10.959894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ab1e-7653-46c9-aca9-c639187b1b91 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:10.976310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:10.978652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:10.979085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:10.979298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.007449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:11.009077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:11.041810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 898/1765] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:11.045946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:11.055257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:11.073787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:11.083548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec4f3-ce83-4808-97d7-3b5dafc95185 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:11.092681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:11.093337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.093741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.094015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.127066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:11.131972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757d18-3178-42e5-a218-db1467e2c86b service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:11.167217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 868/1766] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:11.168751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.174592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.174929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.239119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:11.240940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:11.267613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 869/1767] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:35:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:11.268553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:11.276006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8734020-9500-479b-883b-9bdbd042f2fd service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:11.279228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:11.288965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e0327-747e-4e3e-b9d4-c7472114251a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:11.296579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:11.297400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.297680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.297963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] PATH_INFO: `/v3/users/7ce638d4294c4855a0bdafd37a56171c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.311446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:11.312221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.313481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.317952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.319417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce638d4294c4855a0bdafd37a56171c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:11.330308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:11.457709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce638d4294c4855a0bdafd37a56171c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:11.459357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce638d4294c4855a0bdafd37a56171c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:11.471661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:11.479348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:11.479525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce638d4294c4855a0bdafd37a56171c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:11.498053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 870/1768] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:11.499227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:11.500767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce638d4294c4855a0bdafd37a56171c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:11.528933 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b703-477d-4924-a494-4105126ff607 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:11.540014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:11.544329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.544509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3ad3d12f-5148-4871-b45f-d32e109fc0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce638d4294c4855a0bdafd37a56171c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:11.544966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.545338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.552607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:11.557404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 899/1769] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:11 2022] DELETE /identity/v3/users/7ce638d4294c4855a0bdafd37a56171c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:11.563562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a1e8-f9d9-4820-b9e8-f9c7e728a406 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:11.564081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a1e8-f9d9-4820-b9e8-f9c7e728a406 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:11.564479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a1e8-f9d9-4820-b9e8-f9c7e728a406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:11.592989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:11.596357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:11.626953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9553a1e8-f9d9-4820-b9e8-f9c7e728a406 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:11.628170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9553a1e8-f9d9-4820-b9e8-f9c7e728a406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:11.633596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 871/1770] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:11.748880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 900/1771] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:11.841685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:11.843574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:11.976079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d313fa-b640-4c1d-bf6c-c8de87d36ec3 service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:12.074550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:12.075485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:12.075956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:12.076330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:12.092375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:12.094247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:12.117552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 872/1772] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:12.392012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:12.392502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:12.392573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:12.453229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] MFA Rules not processed for user `eceb9f042cb24343892f09454c7e685a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:12.454468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:12.475831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:12.528869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 901/1773] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:13.340477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:13.341944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:13.369432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a682b-cb1b-48b9-8803-6f3e5bf23794 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:13.385986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:13.387458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:13.387748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:13.388030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] PATH_INFO: `/v3/projects/87b820411401415186a49a29b0099e23` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:13.391273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b820411401415186a49a29b0099e23)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:13.394305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:13.533623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:13.542385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b820411401415186a49a29b0099e23', 'request_id': 'req-f46c436f-7d61-4953-8717-62640e810d95'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.549457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b820411401415186a49a29b0099e23', 'request_id': 'req-f46c436f-7d61-4953-8717-62640e810d95'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.564537 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b820411401415186a49a29b0099e23', 'request_id': 'req-f46c436f-7d61-4953-8717-62640e810d95'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.591896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] The token cache is being invalidate because project 87b820411401415186a49a29b0099e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:13.592156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.592931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f46c436f-7d61-4953-8717-62640e810d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.595079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 873/1774] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:13 2022] DELETE /identity/v3/projects/87b820411401415186a49a29b0099e23 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:13.598527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:13.614927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd79183-3ee3-41ca-84d7-eb4c14a2974a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:13.757919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:13.758636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:13.759895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:13.778971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:13.779866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:13.780147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:13.780420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] PATH_INFO: `/v3/users/1547e6e60aa5435cbc5f4b21ee4adef0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:13.782747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1547e6e60aa5435cbc5f4b21ee4adef0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:13.784672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:13.790137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] MFA Rules not processed for user `dc56e0e37b3d4445a69391142935abcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:13.791359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:13.906416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1547e6e60aa5435cbc5f4b21ee4adef0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.907282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1547e6e60aa5435cbc5f4b21ee4adef0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.920539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 874/1775] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:13.935230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1547e6e60aa5435cbc5f4b21ee4adef0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:13.962123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1547e6e60aa5435cbc5f4b21ee4adef0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:14.002456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a2f25bbd-7976-4264-ad92-0c84e46f0ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1547e6e60aa5435cbc5f4b21ee4adef0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:14.018586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 902/1776] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:13 2022] DELETE /identity/v3/users/1547e6e60aa5435cbc5f4b21ee4adef0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:14.839057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:14.841809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:14.857991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c5f30-eeeb-4a28-9177-96ac02a4fcb7 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:14.991551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:14.992713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:14.993067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:14.993414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] PATH_INFO: `/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:14.995326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4371ac3ac245ebb9fa47c19ecf7a0f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:14.998088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:15.066960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4371ac3ac245ebb9fa47c19ecf7a0f', 'request_id': 'req-d28d4f5b-0989-4291-87ad-b51639185ea3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:15.082241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4371ac3ac245ebb9fa47c19ecf7a0f', 'request_id': 'req-d28d4f5b-0989-4291-87ad-b51639185ea3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:15.096039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4371ac3ac245ebb9fa47c19ecf7a0f', 'request_id': 'req-d28d4f5b-0989-4291-87ad-b51639185ea3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:15.112409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] The token cache is being invalidate because project 8d4371ac3ac245ebb9fa47c19ecf7a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:15.112782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:15.113544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d28d4f5b-0989-4291-87ad-b51639185ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:15.118498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 875/1777] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:14 2022] DELETE /identity/v3/projects/8d4371ac3ac245ebb9fa47c19ecf7a0f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:15.120388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.121219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.121704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.155217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:15.157338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:15.224458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:15.276402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 903/1778] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:15.289887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.291099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.291387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.307389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:15.333521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:15.334808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:15.350832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 876/1779] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:15.352313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:15.364896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06887bbf-02e1-4102-80c2-a833ade29556 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:15.375327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:15.376444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.376875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.377327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.378225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:15.378680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:15.381348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:15.409824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 904/1780] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:15.414355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:15.428353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4670-8e78-40f3-baf7-f5d65085d0f0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:15.440180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:15.479193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.479193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.479193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.483082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:15.486282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:15.533773 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 877/1781] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:15 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:15.542415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:15.573594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f624c7-890e-499c-9571-140c3da8086a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:15.778214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:15.778931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.779212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.779481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.780170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:15.782106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:15.818400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 905/1782] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:15 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:15.819568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:15.838138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f269a31-6caa-404b-a4e0-8a136848ee19 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:15.858023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:15.858721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.859038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.861883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.862662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:15.871425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:15.894749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 878/1783] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:15.896868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:15.910050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b2b8d-160f-4694-a138-f0be014a13fd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:15.941548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:15.962719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:15.963064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:15.963350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:15.965763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:15.967153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686f1de-36c9-456d-9195-e99a4185901d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:16.013138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 906/1784] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:16.018790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:16.064994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146edba-1ca4-488b-90a9-3b18998f52a3 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:16.073002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:16.075003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:16.075851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:16.076326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] PATH_INFO: `/v3/projects/749b5e9137f04e1baf6d5f65112abb9f/users/bef3367af9344f96bb7e85fea2b70920/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:16.101557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749b5e9137f04e1baf6d5f65112abb9f, user_id=bef3367af9344f96bb7e85fea2b70920, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:16.107391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:16.140734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 879/1785] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:16 2022] PUT /identity/v3/projects/749b5e9137f04e1baf6d5f65112abb9f/users/bef3367af9344f96bb7e85fea2b70920/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:16.151665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:16.152482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:16.152482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:16.179683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:16.187631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:16.257279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] MFA Rules not processed for user `bef3367af9344f96bb7e85fea2b70920`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:16.263885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:16.527121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 907/1786] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:16.531905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:16.628353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd4919-3f15-4d19-9830-c739740489b8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:16.645188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:16.650024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:16.652117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:16.652583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] PATH_INFO: `/v3/users/933d48390d03440d9d566c39c134ff21` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:16.655378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933d48390d03440d9d566c39c134ff21)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:16.662175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:16.671782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:16.720571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30975a95-3874-4dfd-ade4-3a4a2978c681 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:16.735491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:16.736401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:16.736827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:16.739783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:16.740797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:16.751080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:16.797949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933d48390d03440d9d566c39c134ff21'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:16.799866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933d48390d03440d9d566c39c134ff21'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:16.800989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933d48390d03440d9d566c39c134ff21'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:16.804037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 908/1787] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:16 2022] POST /identity/v3/projects => generated 375 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:16.807208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:16.811547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:16.824992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933d48390d03440d9d566c39c134ff21'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:16.846453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-72865b5c-dc86-48ca-81b6-1d2f910cb0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933d48390d03440d9d566c39c134ff21'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:16.919717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 880/1788] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:16 2022] DELETE /identity/v3/users/933d48390d03440d9d566c39c134ff21 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:16.921465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:16.923068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:16.923599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:16.958169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:16.960877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:16.980438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:17.008635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f95e-b339-453a-9858-4eb03806c359 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.050959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 881/1789] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:17.068585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.070191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:17.148548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.149071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.150310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.150680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.151695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.171475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.260211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 909/1790] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:16 2022] POST /identity/v3/users => generated 294 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:17.263234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.273413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27083746-f51c-43a8-b6e3-51e33c1f2969 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.284972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.287550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.288488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.289181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.292629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1631740-b487-47c2-b647-85f187b24eff admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.304113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.306227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.306804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.307336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.309391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.316511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.324078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.326394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.358500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 882/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:17.359750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 910/1792] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:17.366054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.404045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73458-f4b3-4a7d-810d-21b930c9bb08 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.414437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.415640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.416072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.416478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] PATH_INFO: `/v3/projects/e29dbad26ae9445597fbc359c328ec11/users/0fa6293bb6b045e88d14d4a28b707b25/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.451961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29dbad26ae9445597fbc359c328ec11, user_id=0fa6293bb6b045e88d14d4a28b707b25, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.455512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.482546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 883/1793] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:17 2022] PUT /identity/v3/projects/e29dbad26ae9445597fbc359c328ec11/users/0fa6293bb6b045e88d14d4a28b707b25/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:17.486387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.507200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843b231-50f4-49e1-b4af-68037501861d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.565050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:17.577584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.578264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.578729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.579042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.579787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.581452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c3643-a621-4a38-9100-a78c6c29e21a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.592658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 911/1794] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:17.596929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.608761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b0b74-2191-4d97-b7ef-0ec5e51cf30c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.613490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.615077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.615846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.615846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] PATH_INFO: `/v3/projects/e29dbad26ae9445597fbc359c328ec11/users/0fa6293bb6b045e88d14d4a28b707b25/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.617987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29dbad26ae9445597fbc359c328ec11, user_id=0fa6293bb6b045e88d14d4a28b707b25, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.620769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.649795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 884/1795] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:17 2022] PUT /identity/v3/projects/e29dbad26ae9445597fbc359c328ec11/users/0fa6293bb6b045e88d14d4a28b707b25/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:17.651187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.651787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.652205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.695145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] MFA Rules not processed for user `0fa6293bb6b045e88d14d4a28b707b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:17.696428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:17.779207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 912/1796] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:17.796615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.798585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:17.904617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643422e-1899-434b-8945-a050df1d37a4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:17.908325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:17.908914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:17.909197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:17.909474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:17.911892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:17.961281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:17.962453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:17.962928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:17.985532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:17.987138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:17.999248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 885/1797] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:18.044265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ede2e-7e26-4054-a8a0-db8402899261 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:18.049434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:18.050385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:18.050887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:18.051259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] PATH_INFO: `/v3/projects/5cde0a93d33f4b06ac523adaff83f412` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:18.052612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cde0a93d33f4b06ac523adaff83f412)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:18.054732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:18.138589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cde0a93d33f4b06ac523adaff83f412', 'request_id': 'req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.146935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cde0a93d33f4b06ac523adaff83f412', 'request_id': 'req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.156592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cde0a93d33f4b06ac523adaff83f412', 'request_id': 'req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.176243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] The token cache is being invalidate because project 5cde0a93d33f4b06ac523adaff83f412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:18.176808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.177626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.180864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 913/1798] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:17 2022] DELETE /identity/v3/projects/5cde0a93d33f4b06ac523adaff83f412 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:18.184534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:18.199523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf1b-1f98-4b56-9370-d28e45c28b5a service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:18.312667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:18.312667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:18.312935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:18.313180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] PATH_INFO: `/v3/users/1e00da4652cd431fb03c55f9174111d3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:18.314588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e00da4652cd431fb03c55f9174111d3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:18.317096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:18.377504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e00da4652cd431fb03c55f9174111d3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.378455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e00da4652cd431fb03c55f9174111d3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.379531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e00da4652cd431fb03c55f9174111d3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.406160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e00da4652cd431fb03c55f9174111d3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.414713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4c7b2b2c-efae-4b6f-a091-002c0af16754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e00da4652cd431fb03c55f9174111d3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.439320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 886/1799] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:18 2022] DELETE /identity/v3/users/1e00da4652cd431fb03c55f9174111d3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:18.745882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:18.747346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:18.757368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:18.762027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cf59-25d2-4db3-b526-80a4677bf8eb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:18.825920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:18.826833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:18.827107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:18.827374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] PATH_INFO: `/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:18.828734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d636b2a7779e4f12a3e1bc8c666e342d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:18.830757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:18.883531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd636b2a7779e4f12a3e1bc8c666e342d', 'request_id': 'req-42f45e9b-3de3-439f-be0a-295f4036844f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.889061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd636b2a7779e4f12a3e1bc8c666e342d', 'request_id': 'req-42f45e9b-3de3-439f-be0a-295f4036844f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.897183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd636b2a7779e4f12a3e1bc8c666e342d', 'request_id': 'req-42f45e9b-3de3-439f-be0a-295f4036844f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.908961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] The token cache is being invalidate because project d636b2a7779e4f12a3e1bc8c666e342d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:18.909248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.909918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-42f45e9b-3de3-439f-be0a-295f4036844f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:18.912366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 914/1800] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:18 2022] DELETE /identity/v3/projects/d636b2a7779e4f12a3e1bc8c666e342d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:18.916979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:18.917275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:18.917607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:18.936785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:18.938004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:18.988975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:18.999181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 887/1801] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:19.006836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.007124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.007397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.025767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:19.026995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:19.034658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 915/1802] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:19.039421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.049099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5ce18-d852-4f70-9bdf-5ca3ecfe00a3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.053905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.054583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.054896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.055174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.055901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:19.056208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.057809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.069124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 888/1803] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:19.073230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.082342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e031d6c-7948-48c7-9b2d-a77e33a90c05 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.087432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.088069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.088351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.088624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.089325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.091386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.120362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 916/1804] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:19.122866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.133369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37dec-11e6-4b7e-b643-f63623c42cf4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.190705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.191343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.191655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.191953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.192674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.194595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.216130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 889/1805] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:19.220055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.231379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dc76c-6009-43bb-9b9e-25262350eefb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.236280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.236992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.237274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.237547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.238282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.240008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.251386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 917/1806] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:19.255517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.266625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3a8de-004e-47c4-b5ed-8c4c9caa9c29 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.272401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.273051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.273329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.273598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.274317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.275950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2c370-3fa7-45b0-b24e-09aa26c6944e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.287111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 890/1807] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:19.291371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.301491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809b38-65b7-45f8-9b84-f19035006980 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.306350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.307274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.307559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.307833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] PATH_INFO: `/v3/projects/c8f7ed04470740049eae4ac4ef7fbd06/users/3226d3a18495487589109d9c942c0244/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.324188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f7ed04470740049eae4ac4ef7fbd06, user_id=3226d3a18495487589109d9c942c0244, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.326986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.343026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 918/1808] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:19 2022] PUT /identity/v3/projects/c8f7ed04470740049eae4ac4ef7fbd06/users/3226d3a18495487589109d9c942c0244/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:19.354372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.354562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.354863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.382164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] MFA Rules not processed for user `3226d3a18495487589109d9c942c0244`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:19.383374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:19.445884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 891/1809] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:19.450386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.484436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:19.515744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16771d0-e62b-469d-b424-c03bb8227d57 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.520983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.521656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.521894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.522167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.522894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.525524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.552766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 919/1810] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:19.557073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.568913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c33ab-2c8d-4ed3-a88b-9c76c529391f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.619956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:19.627605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.628525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.629061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.629570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.630850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.632823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.656514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 892/1811] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:19.657709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.667956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc7a1-fc57-42ab-982c-297dd2cc8e31 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.672620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.673286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.673566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.673841 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.674571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.676212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.686887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 920/1812] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:19.693221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.703953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a63c-e373-4995-a715-d52821d40d8e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.709182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.710234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.710514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.710810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] PATH_INFO: `/v3/projects/849f0c0b60544fa0bb69699adfdd52a9/users/e3c1f1dfddd640939149853dbda305f9/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.726973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=849f0c0b60544fa0bb69699adfdd52a9, user_id=e3c1f1dfddd640939149853dbda305f9, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.730471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.746155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 893/1813] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:19 2022] PUT /identity/v3/projects/849f0c0b60544fa0bb69699adfdd52a9/users/e3c1f1dfddd640939149853dbda305f9/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:19.749960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.760536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12d075-eb16-4536-802f-de9d8d54c7b5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.818192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.818894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.819204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.819532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.820266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.821874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b8c8-117f-4685-9bd4-b749a8d8c9ee admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.832554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 921/1814] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:19.838113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.848207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45a932-5a00-4651-9c9c-9e919d9d1e0e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:19.853305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:19.854177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.854454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.854723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] PATH_INFO: `/v3/projects/849f0c0b60544fa0bb69699adfdd52a9/users/e3c1f1dfddd640939149853dbda305f9/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.856714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849f0c0b60544fa0bb69699adfdd52a9, user_id=e3c1f1dfddd640939149853dbda305f9, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:19.859370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:19.870008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 894/1815] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:19 2022] PUT /identity/v3/projects/849f0c0b60544fa0bb69699adfdd52a9/users/e3c1f1dfddd640939149853dbda305f9/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:19.881588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:19.881877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:19.882156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:19.907855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] MFA Rules not processed for user `e3c1f1dfddd640939149853dbda305f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:19.909056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:19.976230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 922/1816] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:19.986830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:19.988495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:20.045056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8261abc-3564-46aa-b654-5f638b496671 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:20.048622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:20.049248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:20.049524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:20.049791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:20.059411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:20.060815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:20.070612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 895/1817] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:20.140536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:20.150181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a41f45-c8fe-4230-aa92-6b01f825a435 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:20.154017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:20.154683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:20.155009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:20.155384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:20.180371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:20.213040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:20.214478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:20.224587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 923/1818] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:27.361221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:27.378629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2de687-b015-4a5a-bcee-b72a8c0b0030 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:27.398259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:27.402037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:27.402343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:27.402591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:27.412056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:27.522304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:27.524088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d612d1e-2d8b-4044-9655-cb3bb185c5a3 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:27.537453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 896/1819] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:30.220210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=16 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:32.665458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:32.728830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61ed78-6654-4fbc-97cc-7e4388f9dea4 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:32.734050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:32.734992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:32.735276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:32.735670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] PATH_INFO: `/v3/users/3226d3a18495487589109d9c942c0244` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:32.736888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3226d3a18495487589109d9c942c0244)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:32.739399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:32.793052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3226d3a18495487589109d9c942c0244'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:32.793822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3226d3a18495487589109d9c942c0244'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:32.803155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3226d3a18495487589109d9c942c0244'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:32.812669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3226d3a18495487589109d9c942c0244'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:32.825936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3226d3a18495487589109d9c942c0244'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:32.832218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 924/1820] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:32 2022] DELETE /identity/v3/users/3226d3a18495487589109d9c942c0244 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:32.836798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:32.837340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:32.837426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:32.860108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:32.861267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:32.913666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:32.947564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 897/1821] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:32.957699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:32.959327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:33.031207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e45c-faa9-4e04-b3f1-6b6f57307ac0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:33.034928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:33.036183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:33.036460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:33.036736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:33.046759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:33.048316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:33.061800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 925/1822] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:33.393804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:33.396246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:33.417782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1cb78-9d2e-4cda-8cd5-752e5583726c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:33.424908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:33.426125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:33.426560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:33.427000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] PATH_INFO: `/v3/projects/c8f7ed04470740049eae4ac4ef7fbd06` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:33.428367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f7ed04470740049eae4ac4ef7fbd06)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:33.430761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:33.485329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f7ed04470740049eae4ac4ef7fbd06', 'request_id': 'req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.490864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f7ed04470740049eae4ac4ef7fbd06', 'request_id': 'req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.499203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f7ed04470740049eae4ac4ef7fbd06', 'request_id': 'req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.513625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] The token cache is being invalidate because project c8f7ed04470740049eae4ac4ef7fbd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:33.513913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.514799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.516573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 898/1823] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:33 2022] DELETE /identity/v3/projects/c8f7ed04470740049eae4ac4ef7fbd06 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:33.521141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:33.531045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c88f4-4278-426f-8f41-04496c8782b4 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:33.551465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:33.595304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:33.596142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:33.596487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:33.596821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] PATH_INFO: `/v3/users/e3c1f1dfddd640939149853dbda305f9` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:33.598659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c1f1dfddd640939149853dbda305f9)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:33.601071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:33.661444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c1f1dfddd640939149853dbda305f9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.661665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c1f1dfddd640939149853dbda305f9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.670276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c1f1dfddd640939149853dbda305f9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.678411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c1f1dfddd640939149853dbda305f9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.695659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b568605b-6453-4536-95d9-02db73493a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c1f1dfddd640939149853dbda305f9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:33.703821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 926/1824] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:33 2022] DELETE /identity/v3/users/e3c1f1dfddd640939149853dbda305f9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:34.221147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.222835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:34.255554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d3adf-d2e0-4edd-bb4f-d4f5dfb8b447 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.317250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.317988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.318310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.318590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] PATH_INFO: `/v3/projects/849f0c0b60544fa0bb69699adfdd52a9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.320050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849f0c0b60544fa0bb69699adfdd52a9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.322139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.370623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849f0c0b60544fa0bb69699adfdd52a9', 'request_id': 'req-15773ee3-9def-432f-ade6-08246e321d09'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:34.376306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849f0c0b60544fa0bb69699adfdd52a9', 'request_id': 'req-15773ee3-9def-432f-ade6-08246e321d09'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:34.386381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849f0c0b60544fa0bb69699adfdd52a9', 'request_id': 'req-15773ee3-9def-432f-ade6-08246e321d09'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:34.394227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:34.400188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] The token cache is being invalidate because project 849f0c0b60544fa0bb69699adfdd52a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:34.400621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:34.401613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-15773ee3-9def-432f-ade6-08246e321d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:34.403518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 899/1825] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:34 2022] DELETE /identity/v3/projects/849f0c0b60544fa0bb69699adfdd52a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:34.409196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.409618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.409956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.429334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:34.430649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:34.499780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 927/1826] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:34.507519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.508055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.508487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.530018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:34.531372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:34.545005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 900/1827] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:34.545873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.562233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef8dc1-3b02-4ed2-9304-677010fc3ba5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.573921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.574598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.574924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.577152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.578379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:34.579054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.581594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.598375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 928/1828] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:35:34.602609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.619697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4658b-f1df-458d-98f6-f8e0561d0483 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.624826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.625560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.625906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.626247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.627333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.630609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.665104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 901/1829] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:34 2022] POST /identity/v3/projects => generated 407 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:34.666327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.676952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02443f-8f75-44eb-9c21-fafa7c00c841 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.713595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:34.748424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.749025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.749304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.749578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.750263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.752266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.773085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 929/1830] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:34 2022] POST /identity/v3/users => generated 304 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:34.776988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.787133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd838-70d8-4d41-b69e-c9c90fb684ec admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.791708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.792366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.792654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.792926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.793657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.795307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.806527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 902/1831] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:34.810505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.821693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8065-9075-4751-a748-e013d92b18e6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.826523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.827215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.827550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.827823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.828555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.830151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad59cb5-2446-4e70-8eb0-0c651cd6f878 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.840525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 930/1832] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:34.844669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:34.854866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04a039-aa38-4146-92ae-9a457f3c6ddd admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:34.859646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:34.860601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.860881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.861159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] PATH_INFO: `/v3/projects/310006144b5041fc84261f137b69f279/users/186f071a6ae6438283ca7f91f17a54e3/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.876540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310006144b5041fc84261f137b69f279, user_id=186f071a6ae6438283ca7f91f17a54e3, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:34.879778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:34.897413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 903/1833] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:34 2022] PUT /identity/v3/projects/310006144b5041fc84261f137b69f279/users/186f071a6ae6438283ca7f91f17a54e3/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:34.907443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572b47-3162-4f74-9223-687589d612fc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:34.907733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572b47-3162-4f74-9223-687589d612fc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:34.908013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572b47-3162-4f74-9223-687589d612fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:34.934816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-df572b47-3162-4f74-9223-687589d612fc None None] MFA Rules not processed for user `186f071a6ae6438283ca7f91f17a54e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:34.936133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-df572b47-3162-4f74-9223-687589d612fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:35.010002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 931/1834] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:35.010925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.075234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ddccf-f495-4bc0-8977-8c28d65b8884 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.079895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.080552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.080834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.081110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.081813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.083791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.108326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 904/1835] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:35 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:35.112366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572b47-3162-4f74-9223-687589d612fc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.123918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572b47-3162-4f74-9223-687589d612fc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.179485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.180223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.180411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.180682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.181379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.183333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.203733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 932/1836] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:35 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:35.207725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.217779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536c73-a4bb-42ea-ae69-81072475f36c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.222361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.223541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.223825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.224098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.224828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.226677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.237586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 905/1837] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:35.242245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.252345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aea79-ae6a-4af5-9ea4-cef9c5184314 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.257170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.257991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.258267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.258539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] PATH_INFO: `/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b/users/626ab46d4ea34cafab1b70811510c79c/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.273476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3b2d06e1d341e69d52fbdb4e7d2c5b, user_id=626ab46d4ea34cafab1b70811510c79c, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.276208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.289901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 933/1838] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:35 2022] PUT /identity/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b/users/626ab46d4ea34cafab1b70811510c79c/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:35.293872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.304256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d812ec9-1b03-4966-af40-80b4deaa1fac admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.363348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.364599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.364878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.365148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.365871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.367579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939705e-2a63-49ce-bc10-5ce529116f3e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.377145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 906/1839] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:35.381375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.386815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:35.393425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e32f-5760-4b98-9153-9397cfbdf8e5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.398496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.399433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.399768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.400047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] PATH_INFO: `/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b/users/626ab46d4ea34cafab1b70811510c79c/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.402078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3b2d06e1d341e69d52fbdb4e7d2c5b, user_id=626ab46d4ea34cafab1b70811510c79c, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.404753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.415572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 934/1840] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:35 2022] PUT /identity/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b/users/626ab46d4ea34cafab1b70811510c79c/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:35.427305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.427691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.428034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.434801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:35.455998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] MFA Rules not processed for user `626ab46d4ea34cafab1b70811510c79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:35.457214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:35.526115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 907/1841] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:35 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:35.537391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.539165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:35.594171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ed80c-4876-4cff-ad86-3ffc75bc7f0f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.598522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.599196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.599482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.599762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.608885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.610303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.619726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 935/1842] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:35.789419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:35.798708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf7b11-42a1-4c1b-ad68-347325dc1e2b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:35.803186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:35.804126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:35.804411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:35.804689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:35.863204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:35.864676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be83c4c-672f-4788-987a-7587c2139a9b service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:35.873488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 908/1843] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:45.615519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:45.869420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=60 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:49.684677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:49.686824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:49.774165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12478558-9a13-4c6f-984a-624de02a7800 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:49.779497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:49.780216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:49.780647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:49.780976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] PATH_INFO: `/v3/users/124672bb407b41ea99db0bd611f5865c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:49.782376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124672bb407b41ea99db0bd611f5865c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:49.784654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:49.851762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124672bb407b41ea99db0bd611f5865c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:49.852652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124672bb407b41ea99db0bd611f5865c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:49.864463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124672bb407b41ea99db0bd611f5865c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:49.875100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124672bb407b41ea99db0bd611f5865c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:49.899031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124672bb407b41ea99db0bd611f5865c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:49.911397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 936/1844] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:49 2022] DELETE /identity/v3/users/124672bb407b41ea99db0bd611f5865c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:49.912087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:49.912570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:49.912658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:49.932698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:49.934282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.013256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 909/1845] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:50.023665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:50.025215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.090029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e81c-61c9-4bc9-b6d1-1553afa35f90 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:50.094040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:50.094757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:50.095492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:50.095787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:50.107260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:50.108697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:50.120139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 937/1846] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:50.490575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:50.492523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.513993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b429062-eece-4958-93bc-377a45ca1c5f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:50.519712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:50.520806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:50.521098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:50.521527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] PATH_INFO: `/v3/projects/58ef2d58b11443718ffc22e7977b6e57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:50.523171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ef2d58b11443718ffc22e7977b6e57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:50.525274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:50.617499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:50.619198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.641453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ef2d58b11443718ffc22e7977b6e57', 'request_id': 'req-14e0942f-df6b-490a-9061-8291af13413a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.643220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb38ca-19da-4ee1-8193-771ffe5129ee service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:50.649568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ef2d58b11443718ffc22e7977b6e57', 'request_id': 'req-14e0942f-df6b-490a-9061-8291af13413a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.664697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ef2d58b11443718ffc22e7977b6e57', 'request_id': 'req-14e0942f-df6b-490a-9061-8291af13413a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.677018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:50.690443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] The token cache is being invalidate because project 58ef2d58b11443718ffc22e7977b6e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:50.691082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.693012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.697105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 910/1847] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:50 2022] DELETE /identity/v3/projects/58ef2d58b11443718ffc22e7977b6e57 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:50.702828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:50.704490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.741119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:50.742086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:50.742494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:50.742878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] PATH_INFO: `/v3/users/186f071a6ae6438283ca7f91f17a54e3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:50.745039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=186f071a6ae6438283ca7f91f17a54e3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:50.747999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:50.808315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0942f-df6b-490a-9061-8291af13413a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:50.813495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:50.814386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:50.814671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:50.815034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] PATH_INFO: `/v3/users/941bb0b8d3b84a0d9639c8c73aab8230` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:50.816482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186f071a6ae6438283ca7f91f17a54e3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.816963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=941bb0b8d3b84a0d9639c8c73aab8230)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:50.817492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186f071a6ae6438283ca7f91f17a54e3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.818983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:50.829516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186f071a6ae6438283ca7f91f17a54e3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.841112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186f071a6ae6438283ca7f91f17a54e3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.863049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-44e5efac-d037-45cf-9046-112fe40a2c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186f071a6ae6438283ca7f91f17a54e3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.865119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:50.876207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 938/1848] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:50 2022] DELETE /identity/v3/users/186f071a6ae6438283ca7f91f17a54e3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:50.883083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:50.884109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:50.884431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:50.898182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941bb0b8d3b84a0d9639c8c73aab8230'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.899327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941bb0b8d3b84a0d9639c8c73aab8230'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.900258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941bb0b8d3b84a0d9639c8c73aab8230'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.908887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:50.910376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:50.912065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941bb0b8d3b84a0d9639c8c73aab8230'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.926208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941bb0b8d3b84a0d9639c8c73aab8230'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:50.951731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 911/1849] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:50 2022] DELETE /identity/v3/users/941bb0b8d3b84a0d9639c8c73aab8230 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:50.997155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 939/1850] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:51.014134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:51.017362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:51.096617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b453003-502f-4f3a-8bf2-4c774fc0e03f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:51.102590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:51.104045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:51.104045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:51.104045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:51.187047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:51.188757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:51.199289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 912/1851] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:51.590065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:51.591580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:51.605428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f0cb8-f321-4f21-8291-d5668b820ad1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:51.629198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:51.630610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:51.630948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:51.631262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] PATH_INFO: `/v3/projects/310006144b5041fc84261f137b69f279` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:51.639179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310006144b5041fc84261f137b69f279)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:51.641141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:51.687753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:51.700238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310006144b5041fc84261f137b69f279', 'request_id': 'req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.706019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310006144b5041fc84261f137b69f279', 'request_id': 'req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.714508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310006144b5041fc84261f137b69f279', 'request_id': 'req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.726693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] The token cache is being invalidate because project 310006144b5041fc84261f137b69f279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:51.727009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.727676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.729306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 940/1852] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:51 2022] DELETE /identity/v3/projects/310006144b5041fc84261f137b69f279 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:51.733465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:51.743499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b17e-20d2-4a95-b8e7-c4139b19c192 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:51.794194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:51.805619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:51.806578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:51.807039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:51.807314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] PATH_INFO: `/v3/users/626ab46d4ea34cafab1b70811510c79c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:51.808621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626ab46d4ea34cafab1b70811510c79c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:51.810811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:51.845310 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:51.846913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:51.867235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c55398-d65b-4060-8bdc-9e7c8fdb2ff7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:51.872255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626ab46d4ea34cafab1b70811510c79c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.873132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626ab46d4ea34cafab1b70811510c79c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.873287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:51.873880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626ab46d4ea34cafab1b70811510c79c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.874164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:51.874455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:51.874730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] PATH_INFO: `/v3/projects/077e8909fbef424cb254b7718d375136` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:51.875978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=077e8909fbef424cb254b7718d375136)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:51.877910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:51.887810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626ab46d4ea34cafab1b70811510c79c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.897775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-98fc9ef8-a58d-40c4-82fc-680e30432769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626ab46d4ea34cafab1b70811510c79c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.922865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 913/1853] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:51 2022] DELETE /identity/v3/users/626ab46d4ea34cafab1b70811510c79c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:51.940221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077e8909fbef424cb254b7718d375136', 'request_id': 'req-03798e12-03d0-4c74-9142-8aaabc98cdd2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.948240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077e8909fbef424cb254b7718d375136', 'request_id': 'req-03798e12-03d0-4c74-9142-8aaabc98cdd2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.959109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077e8909fbef424cb254b7718d375136', 'request_id': 'req-03798e12-03d0-4c74-9142-8aaabc98cdd2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.976240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] The token cache is being invalidate because project 077e8909fbef424cb254b7718d375136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:51.976561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.977346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-03798e12-03d0-4c74-9142-8aaabc98cdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:51.979165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 941/1854] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:51 2022] DELETE /identity/v3/projects/077e8909fbef424cb254b7718d375136 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:51.983828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:51.984307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:51.984409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.008873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:52.010432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.110958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 914/1855] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:52.117034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.117537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.117816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.137298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:52.138448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.151713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 942/1856] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:52.153701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.164730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2abe5-78b3-468d-a9d7-b6c2702c96e6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.169987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.170681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.170964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.171263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.171978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:52.172291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.174089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.187345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 915/1857] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:52.191567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.201898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94be51-a3d3-47a7-97e1-f1fd9a1cb36d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.208769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.209630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.210014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.210374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.211748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.213812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.255606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 943/1858] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:52.256839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.267202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.268989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.275953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa639-5c0a-4d76-892b-6991018518da admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.389686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57011a2-82b9-40e7-8610-b566e73d9a93 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.391331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.392238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.392818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.393240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.394088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.395193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.396135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.396491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.396718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.397169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] PATH_INFO: `/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.398473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3b2d06e1d341e69d52fbdb4e7d2c5b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.404425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.405260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:52.429134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 916/1859] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:52.432919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.445819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:52.447837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedc119-14a1-4b34-b7af-544fbbdd1422 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.459596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.460590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.462222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.462861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.463772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.466306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.482104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 917/1860] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:52.484680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3b2d06e1d341e69d52fbdb4e7d2c5b', 'request_id': 'req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:52.485845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.492318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3b2d06e1d341e69d52fbdb4e7d2c5b', 'request_id': 'req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:52.504774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8468-a4a6-469b-ab49-c0b1be64afe2 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.515505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3b2d06e1d341e69d52fbdb4e7d2c5b', 'request_id': 'req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:52.528273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] The token cache is being invalidate because project fb3b2d06e1d341e69d52fbdb4e7d2c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:52.528629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:52.529369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69c4885f-5a99-4eb7-95e7-b03a2dd107ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:52.531349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 944/1861] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:52 2022] DELETE /identity/v3/projects/fb3b2d06e1d341e69d52fbdb4e7d2c5b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:52.536620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82556b-1df7-4052-a9de-73f97007eb29 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.536978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82556b-1df7-4052-a9de-73f97007eb29 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.537390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82556b-1df7-4052-a9de-73f97007eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.558995 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4a82556b-1df7-4052-a9de-73f97007eb29 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:52.560457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4a82556b-1df7-4052-a9de-73f97007eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.596103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.596839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.597266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.597551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.598833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.600922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.612401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 918/1862] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:52.616368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.618150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.632125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cf56-43a4-4a8a-8a18-6199a7b43d8d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.659868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.660833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.661106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.661386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] PATH_INFO: `/v3/projects/57f665e54afa4809a9e0ad84c52e9ba2/users/6bcadf577c754d92b7093597498956c2/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.682818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 945/1863] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:52.685609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.687014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.687879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.695492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f665e54afa4809a9e0ad84c52e9ba2, user_id=6bcadf577c754d92b7093597498956c2, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.711705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c894d-6149-4a13-b41b-b7f001b9df2e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.728962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:52.730667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.737088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 919/1864] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:52 2022] PUT /identity/v3/projects/57f665e54afa4809a9e0ad84c52e9ba2/users/6bcadf577c754d92b7093597498956c2/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:52.747752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.748046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.748327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.779187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] MFA Rules not processed for user `6bcadf577c754d92b7093597498956c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:52.780564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:52.810431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 946/1865] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:52.814633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.824634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412031c4-a224-43b3-882d-4241ca8ec1c9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.829413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.830071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.830352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.830624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.831389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:52.831692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.833361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.845390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 947/1866] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:52.850251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.856410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 920/1867] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:52.860925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.861425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8fb-2251-4fbf-8211-154a625dee36 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.866898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.867561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.867846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.868125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.868821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.870740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.872630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ace6-58e6-4078-8a5c-0156f6c88c01 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.877207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.877857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.878140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.878416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.879138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.881658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.903252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 948/1868] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:52.907130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.911801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 921/1869] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:52.915305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:52.931501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:52.937264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:52.983416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e1a-e374-4d6d-a824-a9ae31412d4a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.986156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b37d2-903e-471f-b97b-32b057911d88 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:52.988833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.989611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.989991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.990384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.991129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:52.991303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.991852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:52.992188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:52.992487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:52.993391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:52.993509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:52.995497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.016031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 949/1870] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:53.019635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 922/1871] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:52 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:53.020463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.021658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.031326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf066a-f167-40bb-acf1-a5abf941cebf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.032338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c127f-1dbf-4734-8338-1f990b8d2ccb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.037359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.038013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.038221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.038330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.038553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.038995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.039280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.039364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.039561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.040394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.040899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.042223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.051980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 950/1872] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:53.056075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 923/1873] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:53.056216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.057259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.066158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab6c9-730a-44dd-bc14-f2e300af76e5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.067797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae794818-b3a8-4fb1-890a-a11f39d3aa68 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.071733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.072522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.073184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.073184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.073618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.074164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.075247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.075529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.075808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] PATH_INFO: `/v3/projects/8fe1d62552174e89955e54fc3ddd72f6/users/8f938c5c66754986ab89daf72f3b0ce1/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.076623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.090581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 951/1874] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:53.091786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.094045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe1d62552174e89955e54fc3ddd72f6, user_id=8f938c5c66754986ab89daf72f3b0ce1, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.096950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.116947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341373b-4029-4a22-9c45-f5d036ddb6a1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.119765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 924/1875] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:53 2022] PUT /identity/v3/projects/8fe1d62552174e89955e54fc3ddd72f6/users/8f938c5c66754986ab89daf72f3b0ce1/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:53.123577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.197036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.198737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.199268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.199946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] PATH_INFO: `/v3/projects/c5ea2d2a06694d1fa451cb41b6775635/users/4a49e9a14a2b4e43987e559311628074/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.203333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f242-b1d5-44f8-a7e8-11a046c7237b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.209714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.210378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.210665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.210970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.211706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.213329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.226545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 925/1876] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:35:53.228527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ea2d2a06694d1fa451cb41b6775635, user_id=4a49e9a14a2b4e43987e559311628074, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.230429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.231659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41e86a-0964-4ae0-89fc-ff0a84b680f0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.243815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd505039-2769-46ff-9def-8e933b05be6e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.251209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 952/1877] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:53 2022] PUT /identity/v3/projects/c5ea2d2a06694d1fa451cb41b6775635/users/4a49e9a14a2b4e43987e559311628074/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:53.262902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.263280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.263621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.295628 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] MFA Rules not processed for user `4a49e9a14a2b4e43987e559311628074`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:53.297051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:53.312393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.313286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.313568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.313841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] PATH_INFO: `/v3/projects/8fe1d62552174e89955e54fc3ddd72f6/users/8f938c5c66754986ab89daf72f3b0ce1/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.317209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe1d62552174e89955e54fc3ddd72f6, user_id=8f938c5c66754986ab89daf72f3b0ce1, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.321840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437669b8-4ad2-438a-8872-e7ad0132553f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.335108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 926/1878] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:53 2022] PUT /identity/v3/projects/8fe1d62552174e89955e54fc3ddd72f6/users/8f938c5c66754986ab89daf72f3b0ce1/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:53.346077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.346370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.346652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.360075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:53.362866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:53.380884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 953/1879] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:53.386513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] MFA Rules not processed for user `8f938c5c66754986ab89daf72f3b0ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:53.387747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:53.389025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.390624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:53.460789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3798-8884-4c51-89b5-12ccc3e62dbb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.499051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 927/1880] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:53.513574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.515350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:53.537536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.538408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.539932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.540587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.590795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bad574-5af5-4a0a-9b9b-49966b969c5b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.595248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.595920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.596190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.596466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.610197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.612034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.624058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 928/1881] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:53.659972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.671174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.673399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.693151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 954/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:53.725624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:53.806584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f7f0a-d375-4cdd-8acf-b4f83ca9f9bc service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.812527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.813944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.814229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.814518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] PATH_INFO: `/v3/users/6bcadf577c754d92b7093597498956c2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.815805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bcadf577c754d92b7093597498956c2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.818436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.824572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d1679-3c99-42b3-b8c4-70afdb3b2f66 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:53.842878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:53.843874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.844299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.844691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] PATH_INFO: `/v3/users/4a49e9a14a2b4e43987e559311628074` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.846016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a49e9a14a2b4e43987e559311628074)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:53.848851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:53.860256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:53.872774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:53.893580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bcadf577c754d92b7093597498956c2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.894450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bcadf577c754d92b7093597498956c2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.896008 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bcadf577c754d92b7093597498956c2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.906927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bcadf577c754d92b7093597498956c2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.916016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7cae2cad-c507-45f5-847e-e4c96aaf7abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bcadf577c754d92b7093597498956c2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.927326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a49e9a14a2b4e43987e559311628074'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.928771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a49e9a14a2b4e43987e559311628074'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.941435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 929/1883] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:53 2022] DELETE /identity/v3/users/6bcadf577c754d92b7093597498956c2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:53.943234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a49e9a14a2b4e43987e559311628074'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.946672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.947088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.947577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:53.952942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a49e9a14a2b4e43987e559311628074'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.972456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-25519239-89ba-4712-971a-fd4118bde8d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a49e9a14a2b4e43987e559311628074'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:53.973231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:53.974831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:53.990268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 955/1884] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:53 2022] DELETE /identity/v3/users/4a49e9a14a2b4e43987e559311628074 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:53.990268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:53.990268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:53.990268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.010740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:54.011991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.052304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 930/1885] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:54.062340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.063964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.077883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 956/1886] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:54.091437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.092825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.131716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e0723-24fd-412a-9a60-aad50715b0de None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.165967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddb0ac-2198-4ece-b757-feae51faa44d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.170947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.171910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.172448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.173837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.184591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.186036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.186268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.186961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.187294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.187575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.195977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 957/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:54.201629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.203209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.218578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 931/1888] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:54.540306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.542035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.586950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35980-728f-41d7-a1d8-e88dc3e5bf09 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.618763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.619729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.620007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.620390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] PATH_INFO: `/v3/projects/c5ea2d2a06694d1fa451cb41b6775635` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.621576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ea2d2a06694d1fa451cb41b6775635)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.623537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.651483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.653465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.662447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:54.671888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875f19-c8c8-4cff-8951-c88a8f49accb service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.677768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.679487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.679841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.680207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] PATH_INFO: `/v3/projects/57f665e54afa4809a9e0ad84c52e9ba2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.681796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f665e54afa4809a9e0ad84c52e9ba2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.684055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.687687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:54.691765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ea2d2a06694d1fa451cb41b6775635', 'request_id': 'req-d5b58a36-6918-4f9b-88a8-bdaaed36900a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.697973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ea2d2a06694d1fa451cb41b6775635', 'request_id': 'req-d5b58a36-6918-4f9b-88a8-bdaaed36900a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.706951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ea2d2a06694d1fa451cb41b6775635', 'request_id': 'req-d5b58a36-6918-4f9b-88a8-bdaaed36900a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.720953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] The token cache is being invalidate because project c5ea2d2a06694d1fa451cb41b6775635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:54.721243 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.721899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d5b58a36-6918-4f9b-88a8-bdaaed36900a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.724023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 958/1889] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:54 2022] DELETE /identity/v3/projects/c5ea2d2a06694d1fa451cb41b6775635 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:54.729089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d4c3-bbed-43dd-8d97-29c1ccd6ff17 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.729372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d4c3-bbed-43dd-8d97-29c1ccd6ff17 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.729646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d4c3-bbed-43dd-8d97-29c1ccd6ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.744154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f665e54afa4809a9e0ad84c52e9ba2', 'request_id': 'req-f6e1be1c-b924-44bc-9785-de05cf4510b0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.748756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9e46d4c3-bbed-43dd-8d97-29c1ccd6ff17 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:54.749833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f665e54afa4809a9e0ad84c52e9ba2', 'request_id': 'req-f6e1be1c-b924-44bc-9785-de05cf4510b0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.750378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9e46d4c3-bbed-43dd-8d97-29c1ccd6ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.758986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f665e54afa4809a9e0ad84c52e9ba2', 'request_id': 'req-f6e1be1c-b924-44bc-9785-de05cf4510b0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.770722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] The token cache is being invalidate because project 57f665e54afa4809a9e0ad84c52e9ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:54.771043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.771730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.773278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 932/1890] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:54 2022] DELETE /identity/v3/projects/57f665e54afa4809a9e0ad84c52e9ba2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:54.777022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.778411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.827438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 959/1891] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:54.836097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.836384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.836629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.856945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:54.867301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1be1c-b924-44bc-9785-de05cf4510b0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.871539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:54.872980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.874195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.874592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.875115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] PATH_INFO: `/v3/users/8f938c5c66754986ab89daf72f3b0ce1` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.877132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f938c5c66754986ab89daf72f3b0ce1)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.880039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.892771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 960/1892] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:54.897163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.908020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4a07c-c9a9-4108-9ca6-aab4848369a3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.916672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.916672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.916672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.916672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.917305 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:54.917756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.925906 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.946315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 961/1893] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:35:54.949076 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:54.960199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28ac5-f2ac-45bb-ad12-ae8a8a793a37 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:54.965355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:54.966012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f938c5c66754986ab89daf72f3b0ce1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.966136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:54.966486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:54.966860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:54.966980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f938c5c66754986ab89daf72f3b0ce1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.968130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:54.968235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f938c5c66754986ab89daf72f3b0ce1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.969873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:54.982696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f938c5c66754986ab89daf72f3b0ce1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:54.996965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f938c5c66754986ab89daf72f3b0ce1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.003329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 962/1894] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:54 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:55.008342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.010234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.026431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 933/1895] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:54 2022] DELETE /identity/v3/users/8f938c5c66754986ab89daf72f3b0ce1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:55.116677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab09f-d704-4834-8137-c5e24b91dbe3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.135365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:55.187848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.189398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.191257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.191751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.193244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.197001 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.226629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 963/1896] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:55.227967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.237852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abdfb0-f9e0-430a-a9ce-d8a07f896c64 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.242636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.243426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.244009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.244266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.245052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.246661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.252512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:55.258793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 934/1897] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:55.263431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.274037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375fc55-d23f-434e-916e-ac39399f30f3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.279185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.280014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.280393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.280750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.281584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.283288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.294372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 964/1898] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:55.298676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.308864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d431379-4d25-4a98-b10c-9f26f50c88a7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.313705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.314572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.314874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.315153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] PATH_INFO: `/v3/projects/569ae51482204d47835fe27d4a26e438/users/692115a495cd4af0891616e511db3491/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.330643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569ae51482204d47835fe27d4a26e438, user_id=692115a495cd4af0891616e511db3491, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.333876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91059b-6341-417a-b1b3-30d7dadf6e82 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.338223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.340691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.355221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8028f5-efbe-45c8-8590-00f47ecf1550 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.365380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 935/1899] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:55 2022] PUT /identity/v3/projects/569ae51482204d47835fe27d4a26e438/users/692115a495cd4af0891616e511db3491/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:55.367179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.367479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.367776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.400456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] MFA Rules not processed for user `692115a495cd4af0891616e511db3491`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:55.401662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.414923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.415810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.416091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.416364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] PATH_INFO: `/v3/projects/8fe1d62552174e89955e54fc3ddd72f6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.417648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe1d62552174e89955e54fc3ddd72f6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.420468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.469284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 936/1900] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:55.470339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe1d62552174e89955e54fc3ddd72f6', 'request_id': 'req-c78775bc-3893-43ce-adb9-e6aadc82d71f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.477038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe1d62552174e89955e54fc3ddd72f6', 'request_id': 'req-c78775bc-3893-43ce-adb9-e6aadc82d71f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.481831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.483268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.485034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe1d62552174e89955e54fc3ddd72f6', 'request_id': 'req-c78775bc-3893-43ce-adb9-e6aadc82d71f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.497893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] The token cache is being invalidate because project 8fe1d62552174e89955e54fc3ddd72f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:55.498163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.498923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c78775bc-3893-43ce-adb9-e6aadc82d71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:55.500470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 965/1901] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:55 2022] DELETE /identity/v3/projects/8fe1d62552174e89955e54fc3ddd72f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:55.505274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a6d7-1a70-4b5e-9bba-891f85807975 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.505578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a6d7-1a70-4b5e-9bba-891f85807975 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.505856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a6d7-1a70-4b5e-9bba-891f85807975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.527547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e1e1a6d7-1a70-4b5e-9bba-891f85807975 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:55.528747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e1e1a6d7-1a70-4b5e-9bba-891f85807975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.547456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba4edb-ecfb-4ec8-8f08-ddd60f19d171 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.551587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:55.592762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 966/1902] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:55.597399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.598612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.600931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.600931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.601306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.602118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.602404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.602671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.622047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:55.623120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:55.633434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 967/1903] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:35:55.637585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.639226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:55.656937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766e315-03c6-474f-8a4b-c6427d041c55 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.662272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.663004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.663405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.663692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.664427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:35:55.664735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.666341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.676961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.678550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.684971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 968/1904] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:35:55.689060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 937/1905] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:35:55.689191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.700126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbac26-d273-4265-a411-c0c5e6e3074a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.708013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.708680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.708993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.709274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.710025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.712040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.758462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 969/1906] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:55.759605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.775748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89bdbd-9f2b-487d-b107-d73d55939945 service glance] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.872384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.873135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.873417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.873808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.874616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.876579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.902585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 938/1907] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:55 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:35:55.904143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.924444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e262-bfdf-4aed-8ec3-aaeb042366f4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.930857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.931913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.932307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.932711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.933552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.935514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:55.957976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 970/1908] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:55.963988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:55.977722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ed74a-c232-403d-9e5e-3a40f2784c99 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:55.985630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:55.986413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:55.986702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:55.991497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:55.992499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:55.996034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4d69d-e04c-4b78-b66b-f966c4aa9e71 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.008812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 939/1909] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:56.014267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.027223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294abe-601e-460f-a123-c69dd75795a9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.036772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.037682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.037959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.038232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] PATH_INFO: `/v3/projects/958fadf865034040ab4cbc36d5fc4ee5/users/9866ed46ff7c4f50a963d19ce2a56d5f/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.064138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958fadf865034040ab4cbc36d5fc4ee5, user_id=9866ed46ff7c4f50a963d19ce2a56d5f, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.064138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.081725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 971/1910] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:56 2022] PUT /identity/v3/projects/958fadf865034040ab4cbc36d5fc4ee5/users/9866ed46ff7c4f50a963d19ce2a56d5f/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:56.093390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.093918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.094202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.135119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] MFA Rules not processed for user `9866ed46ff7c4f50a963d19ce2a56d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:56.136769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:56.282960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 940/1911] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:56.290303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.399624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3e3c5-dab6-4a27-adee-020f6464dd29 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.401867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.402535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.402839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.403122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.403829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.405787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.434106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 972/1912] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:35:56 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:56.438415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.450168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa32dd8-4808-45a9-9f15-b24a5e54373d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.502037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:56.520656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.521329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.521605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.521877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.522559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.524502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.544246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 941/1913] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:35:56 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:35:56.548313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.559119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825c200-5562-4f75-8de9-7d5f50d44d98 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.564006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.564660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.564941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.565209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.565971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.567650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.578566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 973/1914] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:56.583140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.592782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730318dd-73fb-4193-9b4e-cd8192743959 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.597310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.598841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.599119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.599387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] PATH_INFO: `/v3/projects/63d515362412467cb8831f6707eb17c0/users/c8dbc48108c24a18875066722afa056d/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.617423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d515362412467cb8831f6707eb17c0, user_id=c8dbc48108c24a18875066722afa056d, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.620175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.633934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 942/1915] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:56 2022] PUT /identity/v3/projects/63d515362412467cb8831f6707eb17c0/users/c8dbc48108c24a18875066722afa056d/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:56.637827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.647625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa69e-0c64-4b3b-b333-1b746be805b1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.666932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:56.704347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.705003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.705281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.705550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.706260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.707911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1462d1-5a1a-43b6-89a9-ecc5a5c02f5d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.717349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 974/1916] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:35:56.721425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.732061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38bb46-bc3c-4388-9f4d-bf806359c636 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.736724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.737605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.737885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.738161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] PATH_INFO: `/v3/projects/63d515362412467cb8831f6707eb17c0/users/c8dbc48108c24a18875066722afa056d/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.740813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d515362412467cb8831f6707eb17c0, user_id=c8dbc48108c24a18875066722afa056d, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:56.743509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:56.754748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 943/1917] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:35:56 2022] PUT /identity/v3/projects/63d515362412467cb8831f6707eb17c0/users/c8dbc48108c24a18875066722afa056d/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:56.766110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.766540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.767250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:56.793230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] MFA Rules not processed for user `c8dbc48108c24a18875066722afa056d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:56.794440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:56.871117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 975/1918] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:56.883489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:56.885182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:56.945506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c5891-af55-4150-84f1-d8a232f20ba8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:56.949239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:56.949887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:56.950169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:56.950445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:57.016480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:57.018141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ed597-21fb-4186-a26a-ce514f6e28d7 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:57.029752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 944/1919] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:58.622146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:58.623689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:58.712697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014b49-d069-4e7e-b35f-d319d2cfb3e8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:58.728942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:58.729852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:58.730168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:58.730481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] PATH_INFO: `/v3/users/bef3367af9344f96bb7e85fea2b70920` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:58.732346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef3367af9344f96bb7e85fea2b70920)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:58.734531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:58.848457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef3367af9344f96bb7e85fea2b70920'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:58.851578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef3367af9344f96bb7e85fea2b70920'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:58.857237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:58.901618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef3367af9344f96bb7e85fea2b70920'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:58.936154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef3367af9344f96bb7e85fea2b70920'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:58.964583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef3367af9344f96bb7e85fea2b70920'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:58.977940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 976/1920] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:58 2022] DELETE /identity/v3/users/bef3367af9344f96bb7e85fea2b70920 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:35:58.993293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:58.998318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:58.999301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:59.046575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:35:59.046575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:59.095432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:35:59.119746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 945/1921] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:35:59.133420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:59.136009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:59.210540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a2c1-61b3-4dc3-9792-560bd34810eb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:59.214627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:59.215391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:59.215739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:59.216082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:59.227283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:59.228814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:59.240672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 977/1922] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:35:59.731628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:59.735756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:35:59.746274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3dcb25-e09a-4476-8c16-b3a843cef7b0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:35:59.752020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:35:59.753053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:35:59.753334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:35:59.753610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] PATH_INFO: `/v3/projects/749b5e9137f04e1baf6d5f65112abb9f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:35:59.755584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=749b5e9137f04e1baf6d5f65112abb9f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:35:59.763582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:35:59.841502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749b5e9137f04e1baf6d5f65112abb9f', 'request_id': 'req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:59.850621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749b5e9137f04e1baf6d5f65112abb9f', 'request_id': 'req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:59.872316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749b5e9137f04e1baf6d5f65112abb9f', 'request_id': 'req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:59.904886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] The token cache is being invalidate because project 749b5e9137f04e1baf6d5f65112abb9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:35:59.904886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:59.905424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:35:59.907195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 946/1923] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:35:59 2022] DELETE /identity/v3/projects/749b5e9137f04e1baf6d5f65112abb9f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:35:59.912141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:35:59.923512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c87f7-b208-4082-9648-5197b0944e55 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:00.038811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:00.039858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:00.040228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:00.041286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] PATH_INFO: `/v3/users/0fa6293bb6b045e88d14d4a28b707b25` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:00.044306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa6293bb6b045e88d14d4a28b707b25)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:00.046486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:00.174337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa6293bb6b045e88d14d4a28b707b25'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:00.175293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa6293bb6b045e88d14d4a28b707b25'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:00.188079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa6293bb6b045e88d14d4a28b707b25'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:00.203816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa6293bb6b045e88d14d4a28b707b25'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:00.218168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:00.256911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-495fd33a-2bcf-459a-a817-3b815490cf25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa6293bb6b045e88d14d4a28b707b25'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:00.274621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 978/1924] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:35:59 2022] DELETE /identity/v3/users/0fa6293bb6b045e88d14d4a28b707b25 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:00.921494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:00.923601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:00.968631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4703-d9df-488a-9f4b-8df0e68e7fbe admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.071961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.073053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.073509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.073791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] PATH_INFO: `/v3/projects/e29dbad26ae9445597fbc359c328ec11` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.075087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29dbad26ae9445597fbc359c328ec11)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.077060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.145272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29dbad26ae9445597fbc359c328ec11', 'request_id': 'req-b294e689-66be-42a6-b422-b8e992896f8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:01.154719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29dbad26ae9445597fbc359c328ec11', 'request_id': 'req-b294e689-66be-42a6-b422-b8e992896f8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:01.168135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29dbad26ae9445597fbc359c328ec11', 'request_id': 'req-b294e689-66be-42a6-b422-b8e992896f8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:01.183896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] The token cache is being invalidate because project e29dbad26ae9445597fbc359c328ec11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:01.183896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:01.184738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b294e689-66be-42a6-b422-b8e992896f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:01.189867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 947/1925] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:00 2022] DELETE /identity/v3/projects/e29dbad26ae9445597fbc359c328ec11 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:01.196032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.196613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.196708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.217776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:01.218452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:01.318978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 979/1926] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:01.326703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.327074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.327460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.351222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:01.379617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:01.381508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:01.394171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 948/1927] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:01.398892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.414302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783907dc-a78e-48f9-8b37-4e4e0207f770 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.420261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.421010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.421405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.421719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.422544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:01.422963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.424728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.438104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 980/1928] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:36:01.442832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.453049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb628-4ac5-4340-ba58-ec179cc116d7 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.462603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.463313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.463600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.463878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.464621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.467385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.512768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 949/1929] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:01 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:01.514252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.525380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801a22-2611-425d-8629-a0368cedbfb1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.585307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.585895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.586178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.586452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.587205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.589131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.628409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.635684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 981/1930] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:01 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:01.640637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ce0c8-88e7-4288-9878-5848e5e3e431 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.645824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.646806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.647252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.647399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.648228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.650137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.689926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 950/1931] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:01.690072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.704816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424989d-75cd-4840-8ed5-d9365f9bcf1d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.720455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.721413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.721413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.721780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.723518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.724240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9e172-d0cb-477e-a967-c87e7d713da5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.741693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 982/1932] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:01.745738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.776713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7f12a-adba-4846-9302-60ff66a2e29b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.784165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.785734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.785944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.786231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] PATH_INFO: `/v3/projects/10784e6d595745608175553e53dffc4b/users/2400191a69f44b549160dc60a5eb84ff/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.810192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10784e6d595745608175553e53dffc4b, user_id=2400191a69f44b549160dc60a5eb84ff, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:01.813189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:01.834825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 951/1933] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:01 2022] PUT /identity/v3/projects/10784e6d595745608175553e53dffc4b/users/2400191a69f44b549160dc60a5eb84ff/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:01.853400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.854017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.854438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.861423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:01.864026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:01.894258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] MFA Rules not processed for user `2400191a69f44b549160dc60a5eb84ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:01.895923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:01.937093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a7f5-742e-443f-a7d6-2f76bfcb9923 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:01.946023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:01.947138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:01.947758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:01.948928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:01.951369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:01.968702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:02.000340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 983/1934] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:02.005787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.028021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.029696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.047054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 952/1935] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:02.083803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c746-244a-4f3f-9293-0c6676d56977 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.090192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.090928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.091507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.091685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.092399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.094461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.124156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 984/1936] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:02 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:02.128295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.140949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe96884-b3bf-400e-86c5-5acb3a465569 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.217913 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:02.231875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.232753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.233075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.233370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.234209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.236378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.264784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 953/1937] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:02 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:02.269372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.280504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ace36-5a04-416a-b75f-7de0aebcc9ba admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.293846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.294561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.294890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.295155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.296005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.297647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.315884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 985/1938] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:02.321437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.368039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b4631-7590-49a0-8e5e-11f8cd04180b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.382617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.383824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.384089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.384371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] PATH_INFO: `/v3/projects/bee2a12274e148568459461c51952d4a/users/c6b12c4f71c749cfaa5ad46cf2ee0d77/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.415229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee2a12274e148568459461c51952d4a, user_id=c6b12c4f71c749cfaa5ad46cf2ee0d77, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.418499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.448885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 954/1939] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:02 2022] PUT /identity/v3/projects/bee2a12274e148568459461c51952d4a/users/c6b12c4f71c749cfaa5ad46cf2ee0d77/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:02.452746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.466282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc2fbf-4ee5-488e-acc4-390c33dfdf38 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.566754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.567551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.567840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.568213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.568908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.570910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8068d3c-4e97-4e49-9543-4963719865b2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.591327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 986/1940] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:02.592568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.607668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b3f8-4af5-4c98-9462-96ab05d763a8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.613889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.615071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.615263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.615613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] PATH_INFO: `/v3/projects/bee2a12274e148568459461c51952d4a/users/c6b12c4f71c749cfaa5ad46cf2ee0d77/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.617724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee2a12274e148568459461c51952d4a, user_id=c6b12c4f71c749cfaa5ad46cf2ee0d77, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.620675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.638906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 955/1941] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:02 2022] PUT /identity/v3/projects/bee2a12274e148568459461c51952d4a/users/c6b12c4f71c749cfaa5ad46cf2ee0d77/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:02.652366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.652366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.652366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.698387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] MFA Rules not processed for user `c6b12c4f71c749cfaa5ad46cf2ee0d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:02.699903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:02.755382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:02.798889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 987/1942] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:02.811271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:02.813677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:02.897704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a402356-acdb-4dd8-91bd-1efb00bb45bb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:02.901674 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:02.902556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:02.902741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:02.903054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:02.928844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:02.931477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:02.944502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 956/1943] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:03.065059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:03.076958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dca9e-15ee-49de-9121-4ffd888ca368 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:03.081121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:03.081886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:03.082832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:03.083218 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:03.168236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:03.169695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc8f94-e861-4a84-a13e-2e7ccbfb9f64 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:03.179837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 988/1944] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:03.543928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:03.629590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468a0ae-6580-4e1c-ba80-eb6a4b5f49bf service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:03.634534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:03.635555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:03.635840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:03.636125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] PATH_INFO: `/v3/users/2400191a69f44b549160dc60a5eb84ff` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:03.637363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2400191a69f44b549160dc60a5eb84ff)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:03.644056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:03.705421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2400191a69f44b549160dc60a5eb84ff'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:03.706163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2400191a69f44b549160dc60a5eb84ff'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:03.706926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2400191a69f44b549160dc60a5eb84ff'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:03.709421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:03.718170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2400191a69f44b549160dc60a5eb84ff'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:03.729645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2400191a69f44b549160dc60a5eb84ff'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:03.751023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 957/1945] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:03 2022] DELETE /identity/v3/users/2400191a69f44b549160dc60a5eb84ff => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:03.756072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:03.756575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:03.756652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:03.779398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:03.780898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:03.846853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 989/1946] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:03.857596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:03.859359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:03.921854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b8f24-c294-45f2-b3da-c9552350aab8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:03.925472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:03.926706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:03.927029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:03.927322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:03.936641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:03.938086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:03.952651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 958/1947] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:04.330654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:04.335062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:04.352620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4319f-22bf-4658-8b73-d4daa0d35848 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:04.358850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:04.359759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:04.360054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:04.360354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] PATH_INFO: `/v3/projects/10784e6d595745608175553e53dffc4b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:04.362055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10784e6d595745608175553e53dffc4b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:04.364369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:04.430064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10784e6d595745608175553e53dffc4b', 'request_id': 'req-34e1e1c4-de6f-4e62-bf63-542464f534c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.436037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10784e6d595745608175553e53dffc4b', 'request_id': 'req-34e1e1c4-de6f-4e62-bf63-542464f534c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.444753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10784e6d595745608175553e53dffc4b', 'request_id': 'req-34e1e1c4-de6f-4e62-bf63-542464f534c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.453813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:04.461481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] The token cache is being invalidate because project 10784e6d595745608175553e53dffc4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:04.462025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.463650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.465025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 990/1948] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:04 2022] DELETE /identity/v3/projects/10784e6d595745608175553e53dffc4b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:04.468966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:04.479233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a2416-ade9-41a8-91d3-b26c56636079 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:04.540960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:04.541841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:04.542129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:04.542429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] PATH_INFO: `/v3/users/c6b12c4f71c749cfaa5ad46cf2ee0d77` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:04.543717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b12c4f71c749cfaa5ad46cf2ee0d77)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:04.545706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:04.600196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b12c4f71c749cfaa5ad46cf2ee0d77'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.601195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b12c4f71c749cfaa5ad46cf2ee0d77'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.601802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b12c4f71c749cfaa5ad46cf2ee0d77'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.609576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b12c4f71c749cfaa5ad46cf2ee0d77'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.618361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b12c4f71c749cfaa5ad46cf2ee0d77'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.624276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:04.644379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 959/1949] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:04 2022] DELETE /identity/v3/users/c6b12c4f71c749cfaa5ad46cf2ee0d77 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:04.754479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:04.756249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:04.783532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e1c4-de6f-4e62-bf63-542464f534c0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:04.859502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:04.860388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:04.860670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:04.860939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] PATH_INFO: `/v3/users/7ba112db5e4547f5bae76ab64e5cb23c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:04.862319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba112db5e4547f5bae76ab64e5cb23c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:04.864983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:04.945699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba112db5e4547f5bae76ab64e5cb23c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.947655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba112db5e4547f5bae76ab64e5cb23c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.955918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba112db5e4547f5bae76ab64e5cb23c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.964180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:04.965259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba112db5e4547f5bae76ab64e5cb23c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.966715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:04.984284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f882cc-9d43-4a8b-931a-a1bb5814ccf0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:04.987026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b8fd594f-6321-4b60-95d8-9d09e2ef8b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba112db5e4547f5bae76ab64e5cb23c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:04.992419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:04.992991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:04.993223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:04.993543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] PATH_INFO: `/v3/projects/bee2a12274e148568459461c51952d4a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:04.995397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee2a12274e148568459461c51952d4a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:04.997484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:04.997877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 991/1950] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:04 2022] DELETE /identity/v3/users/7ba112db5e4547f5bae76ab64e5cb23c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:05.045047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee2a12274e148568459461c51952d4a', 'request_id': 'req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.051554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee2a12274e148568459461c51952d4a', 'request_id': 'req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.059124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee2a12274e148568459461c51952d4a', 'request_id': 'req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.071514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] The token cache is being invalidate because project bee2a12274e148568459461c51952d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:05.072227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.072452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7567ee5d-0d84-4072-b38e-d03bc2f98e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.074200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 960/1951] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:04 2022] DELETE /identity/v3/projects/bee2a12274e148568459461c51952d4a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:05.079874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.080167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.080595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.100801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.102217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.169424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 992/1952] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:05.178094 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.178430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.178715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.189028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.191244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.198716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.200259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.207987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657a8ab-507b-432f-9601-bb4f3dc89728 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.209332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 961/1953] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:05.214307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.217473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.218652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.219225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.219629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] PATH_INFO: `/v3/projects/42c353894cf64e6ca9e2e4311632eef4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.221279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c353894cf64e6ca9e2e4311632eef4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.224234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.227210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afc00e-bded-47b3-8211-1b6acbfede19 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.231998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.232661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.232949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.233224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.233949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:05.234262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.235888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.242351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:05.249126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 962/1954] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:05.253654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.265341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f596ff-bbf5-422e-ac88-501e91d880f6 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.276323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c353894cf64e6ca9e2e4311632eef4', 'request_id': 'req-d607660e-6691-443f-903b-f8885165eba1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.282107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c353894cf64e6ca9e2e4311632eef4', 'request_id': 'req-d607660e-6691-443f-903b-f8885165eba1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.295803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c353894cf64e6ca9e2e4311632eef4', 'request_id': 'req-d607660e-6691-443f-903b-f8885165eba1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.308285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] The token cache is being invalidate because project 42c353894cf64e6ca9e2e4311632eef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:05.308940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.309229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d607660e-6691-443f-903b-f8885165eba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:05.310822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 993/1955] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:05 2022] DELETE /identity/v3/projects/42c353894cf64e6ca9e2e4311632eef4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:05.315554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a703d5-7436-4bbf-a0c3-62576b5d7bd8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.315819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a703d5-7436-4bbf-a0c3-62576b5d7bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.316128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a703d5-7436-4bbf-a0c3-62576b5d7bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.330413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.331116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.331405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.331677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.332398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.334347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.335514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e0a703d5-7436-4bbf-a0c3-62576b5d7bd8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.336715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e0a703d5-7436-4bbf-a0c3-62576b5d7bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.351857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 994/1956] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:05.353820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.354161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.354504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.366057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 963/1957] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:05.370096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.371754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.374977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.376220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.417521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:05.449650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2b1cd-6df0-47d9-9a72-45f3ef744de7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.458212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.459144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.459599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.459923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 995/1958] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:05.460179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.460889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.462865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.464052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.474074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f9636-9e87-44c9-a288-f75914a1613e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.479317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.479989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.480264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.480539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.481275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:05.481583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.484039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.494821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 964/1959] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:05.495985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.500002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 996/1960] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:36:05.504510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.514896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dadac-dcf0-4dcf-9adb-ee86d30374c4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.515760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a1f2b-4fbf-4dd4-9b41-10bc28548dda admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.520593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.520733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.521302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.521583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.524815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.524937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.524937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.525095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.525166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.525166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.525289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.525638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.538218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 965/1961] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:05.539491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.551600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b210e5-ce71-42d2-9be9-ede0e9712d6a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.561264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 997/1962] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:05.561490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.561490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.561490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.561490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.561780 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.562293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.562409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.575199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 966/1963] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:05.579509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.590157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b362-113d-4098-94f4-7d0f6efb6e84 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.639901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02c1f-7da0-463b-8db8-ad0e114c0c22 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.645210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.645876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.646162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.646435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.647163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.649082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.653199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.654184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.654467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.654741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] PATH_INFO: `/v3/projects/224fb6d4602149ac9b35d7e032359b83/users/f1104d1863c24a9c9f1e548041360bf4/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.656729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:05.673239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=224fb6d4602149ac9b35d7e032359b83, user_id=f1104d1863c24a9c9f1e548041360bf4, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.677519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 998/1964] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:05.677643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7fdb7-5d59-4333-9c98-84d65e15fbdd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.678524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.688379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef3b3-87b6-46be-ae83-1044abb83f9d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.689760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 967/1965] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:05 2022] PUT /identity/v3/projects/224fb6d4602149ac9b35d7e032359b83/users/f1104d1863c24a9c9f1e548041360bf4/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:05.701230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.701452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.701730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.731803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] MFA Rules not processed for user `f1104d1863c24a9c9f1e548041360bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.733350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.747066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.747740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.748026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.748373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.749013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.751049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.764634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 999/1966] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:05.765620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.775534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a8b1d-4fb7-45c8-923b-d1cfb7e11b5e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.780136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.780776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.781055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.781328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.782041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.783661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.797482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1000/1967] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:05.798492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.806403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 968/1968] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:05.810491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed623e20-ba4e-4621-9d25-5806045bc6db admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.814273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.815188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.815497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.815772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] PATH_INFO: `/v3/projects/dec5f9fd916142e3b67cfe76fd79446d/users/7a7be6dec1d344c685ccc8d1f33e73f0/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.818191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:05.819620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.833115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec5f9fd916142e3b67cfe76fd79446d, user_id=7a7be6dec1d344c685ccc8d1f33e73f0, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:05.835824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3ea16-487a-41ce-83bf-4cf43195ac67 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:05.838841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:05.851247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1001/1969] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:05 2022] PUT /identity/v3/projects/dec5f9fd916142e3b67cfe76fd79446d/users/7a7be6dec1d344c685ccc8d1f33e73f0/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:05.864002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.864374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.864779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.882016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e204b-dea8-427d-bbac-c1836105a619 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:05.891048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] MFA Rules not processed for user `7a7be6dec1d344c685ccc8d1f33e73f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:05.892254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:05.934856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:05.935554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:05.935862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:05.936163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:05.978653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1002/1970] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:05.983094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.005989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.007471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.017501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 969/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:06.043024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:06.047084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.073367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24774b58-729d-4d75-a2e7-d94e043e282c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.088918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.088918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.088918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.088918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.089334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.089334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.117136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1003/1972] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:05 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:06.117696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c69c9-bfbe-487d-b94b-3b9aebf93462 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.118365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.156718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:06.186228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.187131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.187405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.187675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] PATH_INFO: `/v3/users/f1104d1863c24a9c9f1e548041360bf4` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.188869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1104d1863c24a9c9f1e548041360bf4)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.190388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685b1d7-c359-4a35-ae20-c1184c12e48c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.190801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.195830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.196495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.196793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.197163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.197860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.199815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.220783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1004/1973] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:06 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:06.224540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.234636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3a1d-538b-4487-8427-e0af6e7f2807 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.237957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1104d1863c24a9c9f1e548041360bf4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:06.239147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1104d1863c24a9c9f1e548041360bf4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:06.239765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1104d1863c24a9c9f1e548041360bf4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:06.240500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.241181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.241466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.241738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.242501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.244650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.247582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1104d1863c24a9c9f1e548041360bf4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:06.255355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-da67a603-738b-4a50-94a9-72a9a9d4d298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1104d1863c24a9c9f1e548041360bf4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:06.256185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1005/1974] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:06.260032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.261768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.273291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 970/1975] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:06 2022] DELETE /identity/v3/users/f1104d1863c24a9c9f1e548041360bf4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:06.275715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e87152-8d13-4840-9889-ba7ca5c03c1a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.277665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.277945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.278220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.296541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:06.298070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.338711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.339632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.339915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.340224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] PATH_INFO: `/v3/projects/c2dd3a1ff3f14657914fd6b57649e354/users/209298d018ef45adb5a399f6c2c2ce27/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.357165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2dd3a1ff3f14657914fd6b57649e354, user_id=209298d018ef45adb5a399f6c2c2ce27, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.359888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.367934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 971/1976] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:06.377778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.378689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1006/1977] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:06 2022] PUT /identity/v3/projects/c2dd3a1ff3f14657914fd6b57649e354/users/209298d018ef45adb5a399f6c2c2ce27/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:06.380727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.383020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.415190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:06.444650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523f64c-75b6-49eb-962c-de796dcb98c5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.450297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:06.451991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5cae9-90a6-4183-8249-781793620ca0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.458072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.459020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.459362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.459591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.460468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.461925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.475809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1007/1978] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:06.479684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.492320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb6f0c-7058-49ab-8396-0e66b419e6d5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.499196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.500260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.500549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.500831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] PATH_INFO: `/v3/projects/c2dd3a1ff3f14657914fd6b57649e354/users/209298d018ef45adb5a399f6c2c2ce27/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.503399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2dd3a1ff3f14657914fd6b57649e354, user_id=209298d018ef45adb5a399f6c2c2ce27, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.506284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17125ee8-1e44-4019-b2e8-96366751a00b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.516814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.517137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1008/1979] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:06 2022] PUT /identity/v3/projects/c2dd3a1ff3f14657914fd6b57649e354/users/209298d018ef45adb5a399f6c2c2ce27/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:06.519177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.520035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.520389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.528598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.529048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.529328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.530407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.531916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.542499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 972/1980] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:06.561669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] MFA Rules not processed for user `209298d018ef45adb5a399f6c2c2ce27`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:06.563062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.657037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1009/1981] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:06.668354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.669760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.738901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76b15-d1a4-4c86-9866-ef69dda94fbb service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.804197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.805058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.805340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.805791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.815926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.817667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.828157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 973/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:06.904399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:06.917797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640d90-f59d-4533-bcae-f16c485281c9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:06.921965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:06.922619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:06.922948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:06.923222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:06.925091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:06.988619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:06.990327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:06.999758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1010/1983] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:07.050222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:07.052160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:07.122438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:07.127139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:07.144987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a8c1f-07fe-423a-9b93-226704635461 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:07.151040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:07.152202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:07.152412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:07.152757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] PATH_INFO: `/v3/projects/224fb6d4602149ac9b35d7e032359b83` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:07.154516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224fb6d4602149ac9b35d7e032359b83)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:07.159120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:07.172771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:07.187623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:07.216589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224fb6d4602149ac9b35d7e032359b83', 'request_id': 'req-c74f0034-b627-43f5-a650-91186d5e6281'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.226273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c139-6073-4e5a-8b00-950433894b8c service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:07.233023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224fb6d4602149ac9b35d7e032359b83', 'request_id': 'req-c74f0034-b627-43f5-a650-91186d5e6281'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.246057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224fb6d4602149ac9b35d7e032359b83', 'request_id': 'req-c74f0034-b627-43f5-a650-91186d5e6281'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.258418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] The token cache is being invalidate because project 224fb6d4602149ac9b35d7e032359b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:07.258750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.259419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c74f0034-b627-43f5-a650-91186d5e6281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.260973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 974/1984] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:07 2022] DELETE /identity/v3/projects/224fb6d4602149ac9b35d7e032359b83 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:07.289962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:07.290893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:07.291175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:07.291453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] PATH_INFO: `/v3/users/9866ed46ff7c4f50a963d19ce2a56d5f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:07.292869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9866ed46ff7c4f50a963d19ce2a56d5f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:07.294922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:07.352062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9866ed46ff7c4f50a963d19ce2a56d5f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.352764 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9866ed46ff7c4f50a963d19ce2a56d5f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.361039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9866ed46ff7c4f50a963d19ce2a56d5f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.372449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9866ed46ff7c4f50a963d19ce2a56d5f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.387204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9866ed46ff7c4f50a963d19ce2a56d5f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.394985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1011/1985] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:07 2022] DELETE /identity/v3/users/9866ed46ff7c4f50a963d19ce2a56d5f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:07.399882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:07.400319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:07.400694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:07.423337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:07.424571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:07.489954 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 975/1986] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:07.499880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:07.501221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:07.560225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dd700-8351-4f24-a526-6a8831003737 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:07.563921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:07.564569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:07.564848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:07.565119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:07.574821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:07.576246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:07.586297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1012/1987] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:07.861382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:07.863074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:07.883830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe503f-1dae-4a8d-8acd-2ff9c39459f6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:07.894919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:07.895877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:07.896225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:07.896504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] PATH_INFO: `/v3/projects/958fadf865034040ab4cbc36d5fc4ee5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:07.898153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958fadf865034040ab4cbc36d5fc4ee5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:07.900194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:07.953765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958fadf865034040ab4cbc36d5fc4ee5', 'request_id': 'req-78923a3c-e750-4b6f-9e8e-a782e335ca99'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.961284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958fadf865034040ab4cbc36d5fc4ee5', 'request_id': 'req-78923a3c-e750-4b6f-9e8e-a782e335ca99'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.973377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958fadf865034040ab4cbc36d5fc4ee5', 'request_id': 'req-78923a3c-e750-4b6f-9e8e-a782e335ca99'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.991875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] The token cache is being invalidate because project 958fadf865034040ab4cbc36d5fc4ee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:07.992240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.993149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:07.994759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 976/1988] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:07 2022] DELETE /identity/v3/projects/958fadf865034040ab4cbc36d5fc4ee5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:08.004869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:08.011452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:08.016072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d4c25-3463-4569-99f8-2466fd43f6ed service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:08.074215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:08.075042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:08.075323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:08.075598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] PATH_INFO: `/v3/users/c8dbc48108c24a18875066722afa056d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:08.076780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8dbc48108c24a18875066722afa056d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:08.078696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:08.131568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8dbc48108c24a18875066722afa056d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.132226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8dbc48108c24a18875066722afa056d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.139214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8dbc48108c24a18875066722afa056d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.146968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8dbc48108c24a18875066722afa056d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.159810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b0a85cf-75c6-40d1-83cc-40aa5add457d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8dbc48108c24a18875066722afa056d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.165168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1013/1989] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:08 2022] DELETE /identity/v3/users/c8dbc48108c24a18875066722afa056d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:08.663620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:08.665077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:08.678244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923a3c-e750-4b6f-9e8e-a782e335ca99 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:08.700563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:08.740270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:08.741128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:08.741408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:08.741677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] PATH_INFO: `/v3/projects/63d515362412467cb8831f6707eb17c0` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:08.742898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d515362412467cb8831f6707eb17c0)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:08.744792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:08.792056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d515362412467cb8831f6707eb17c0', 'request_id': 'req-84f12df9-89f9-4309-8f63-ecdc9342f225'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.797434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d515362412467cb8831f6707eb17c0', 'request_id': 'req-84f12df9-89f9-4309-8f63-ecdc9342f225'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.804631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d515362412467cb8831f6707eb17c0', 'request_id': 'req-84f12df9-89f9-4309-8f63-ecdc9342f225'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.815612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] The token cache is being invalidate because project 63d515362412467cb8831f6707eb17c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:08.815894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.816549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-84f12df9-89f9-4309-8f63-ecdc9342f225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:08.818106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 977/1990] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:08 2022] DELETE /identity/v3/projects/63d515362412467cb8831f6707eb17c0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:08.824016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:08.824306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:08.824610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:08.842447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:08.843647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:08.903563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1014/1991] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:08.914468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:08.914753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:08.915064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:08.933401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:08.935445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:08.945103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 978/1992] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:08.949307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:08.959734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2218a-ed17-4ba8-9452-db858ac81a6f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:08.964259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:08.964901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:08.965174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:08.965444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:08.966163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:08.966466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:08.968094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:08.980245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1015/1993] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:08.985719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:08.995592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b1c87-9dc2-4551-b26b-d91e8c0f823a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.000313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.000982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.001260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.001532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.002235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.004316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.031379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 979/1994] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:08 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:09.036175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:09.047420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde10-1337-4477-9d40-c668e22d7a2d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.056862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:09.113672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.114344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.114624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.114948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.115787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.117746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.141846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1016/1995] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:09 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:09.143141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:09.153969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0f28-c6f5-406d-a017-b87766d3def5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.159329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.161382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.161670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.162149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.163241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.165392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.180297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 980/1996] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:09.181701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:09.192103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83c299-659b-415d-bedd-1e7f7cc5938c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.196790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.197518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.197880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.198163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.199151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.201307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce52fa8-6536-4035-8c20-65273aadcb68 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.212851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1017/1997] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:09.216984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:09.227343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378d280-a494-4493-ba7e-d7ee682eebbd admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.232693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.233590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.233863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.234130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] PATH_INFO: `/v3/projects/471779ee2c1f458ca81bfa5f042ecca6/users/2a06eee0b15b494b8c0e41b33fb75f07/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.249353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471779ee2c1f458ca81bfa5f042ecca6, user_id=2a06eee0b15b494b8c0e41b33fb75f07, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.252042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.266721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 981/1998] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:09 2022] PUT /identity/v3/projects/471779ee2c1f458ca81bfa5f042ecca6/users/2a06eee0b15b494b8c0e41b33fb75f07/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:09.278941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.279236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.279517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.309008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] MFA Rules not processed for user `2a06eee0b15b494b8c0e41b33fb75f07`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:09.310275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:09.377607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1018/1999] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:09.390009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:09.391757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:09.450237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5ac3-77ff-4627-8539-ec6779145af1 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:09.453962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:09.454606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:09.454946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:09.455224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:09.460508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:09.465387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:09.467912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d7046-d68b-4e4a-8c24-d95f00031d06 service glance] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:09.478070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 982/2000] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:16.940839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:16.942288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:17.021010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46b0f0-4127-4d82-8429-50884b4fecf6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:17.025875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:17.027005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:17.027298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:17.027573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] PATH_INFO: `/v3/users/7a7be6dec1d344c685ccc8d1f33e73f0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:17.029014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7be6dec1d344c685ccc8d1f33e73f0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:17.031020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:17.053042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:17.079883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7be6dec1d344c685ccc8d1f33e73f0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.080536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7be6dec1d344c685ccc8d1f33e73f0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.088268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7be6dec1d344c685ccc8d1f33e73f0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.095339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7be6dec1d344c685ccc8d1f33e73f0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.107909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7be6dec1d344c685ccc8d1f33e73f0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.114637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1019/2001] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:16 2022] DELETE /identity/v3/users/7a7be6dec1d344c685ccc8d1f33e73f0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:17.119160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:17.119650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:17.119746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:17.137059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:17.138254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:17.203633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 983/2002] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:17.214141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:17.215613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:17.288060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3736-e758-4aba-b76d-f8836bdb8fd6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:17.292716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:17.293375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:17.293652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:17.293922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:17.304082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:17.305519 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:17.315876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1020/2003] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:17.641216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:17.643057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:17.666024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ab2d0-5c4f-46fa-96dc-7505705aee78 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:17.675958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:17.676835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:17.677118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:17.677402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] PATH_INFO: `/v3/projects/dec5f9fd916142e3b67cfe76fd79446d` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:17.678593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec5f9fd916142e3b67cfe76fd79446d)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:17.680590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:17.752023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec5f9fd916142e3b67cfe76fd79446d', 'request_id': 'req-0c1f1584-2f52-4c5d-a37f-343046611065'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.758846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec5f9fd916142e3b67cfe76fd79446d', 'request_id': 'req-0c1f1584-2f52-4c5d-a37f-343046611065'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.768725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec5f9fd916142e3b67cfe76fd79446d', 'request_id': 'req-0c1f1584-2f52-4c5d-a37f-343046611065'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.781079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] The token cache is being invalidate because project dec5f9fd916142e3b67cfe76fd79446d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:17.781353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.782074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.783740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 984/2004] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:17 2022] DELETE /identity/v3/projects/dec5f9fd916142e3b67cfe76fd79446d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:17.787742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:17.799704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000cda-f610-46d8-9638-7c1f3f44a33c service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:17.876562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:17.877330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:17.877614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:17.877890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] PATH_INFO: `/v3/users/209298d018ef45adb5a399f6c2c2ce27` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:17.879208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] RBAC: Authorizing `identity:delete_user(user_id=209298d018ef45adb5a399f6c2c2ce27)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:17.881292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:17.935424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:17.937989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209298d018ef45adb5a399f6c2c2ce27'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.939002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209298d018ef45adb5a399f6c2c2ce27'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.946724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209298d018ef45adb5a399f6c2c2ce27'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.954035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209298d018ef45adb5a399f6c2c2ce27'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.965975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-978a79b7-6255-4d02-997c-af62750809da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209298d018ef45adb5a399f6c2c2ce27'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:17.975070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1021/2005] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:17 2022] DELETE /identity/v3/users/209298d018ef45adb5a399f6c2c2ce27 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:18.250113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.251574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:18.268320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1584-2f52-4c5d-a37f-343046611065 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.332705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.333575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.333880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.334150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] PATH_INFO: `/v3/projects/c2dd3a1ff3f14657914fd6b57649e354` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.335365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2dd3a1ff3f14657914fd6b57649e354)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.337251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.358955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:18.385491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2dd3a1ff3f14657914fd6b57649e354', 'request_id': 'req-d912bc67-bb03-404c-b3ab-b796870480fd'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:18.390951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2dd3a1ff3f14657914fd6b57649e354', 'request_id': 'req-d912bc67-bb03-404c-b3ab-b796870480fd'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:18.398376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2dd3a1ff3f14657914fd6b57649e354', 'request_id': 'req-d912bc67-bb03-404c-b3ab-b796870480fd'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:18.409237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] The token cache is being invalidate because project c2dd3a1ff3f14657914fd6b57649e354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:18.409546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:18.410903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d912bc67-bb03-404c-b3ab-b796870480fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:18.412600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 985/2006] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:18 2022] DELETE /identity/v3/projects/c2dd3a1ff3f14657914fd6b57649e354 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:18.418763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.419067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.419370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.437794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:18.439023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:18.501529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1022/2007] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:18.509221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.509509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.509786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.528531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:18.529728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:18.538415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 986/2008] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:18.542345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.551929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e2ed8-e7a9-487a-971d-6349917ea06e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.556737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.557394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.557678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.557957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.558720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:18.559089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.560697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.574882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1023/2009] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:18.577360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.590605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d534a-c2d7-42ef-8799-25d5f0826f97 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.595667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.596325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.596608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.596878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.597567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.599500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.624832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 987/2010] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:18.628937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.639402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4f3d5-853f-47b6-a860-85ec7e28e91c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.694202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.694905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.695196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.695475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.696179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.698098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.719349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1024/2011] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:18.723889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.734513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e26e9-f44e-4062-9641-f395adf18839 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.739772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.740423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.740704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.740975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.741822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.743485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.756846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 988/2012] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:18.758065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.768246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf5f1-1dd9-49af-96c4-d94ac3062239 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.773006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.773664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.773944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.774221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.774993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.776581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4efa-3bb7-4454-ac09-aed13c986d75 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.787563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1025/2013] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:18.791510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:18.803323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4232a8-fba1-4e4c-a693-548478062e00 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:18.808078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:18.808959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.809229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.809639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] PATH_INFO: `/v3/projects/183934503b5a405397a2d721701930c2/users/1b17b4c054a446ecb33c62bafa16cec5/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.826226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=183934503b5a405397a2d721701930c2, user_id=1b17b4c054a446ecb33c62bafa16cec5, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:18.828917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:18.841769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 989/2014] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:18 2022] PUT /identity/v3/projects/183934503b5a405397a2d721701930c2/users/1b17b4c054a446ecb33c62bafa16cec5/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:18.853654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:18.853979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:18.854258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:18.881183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] MFA Rules not processed for user `1b17b4c054a446ecb33c62bafa16cec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:18.882414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:18.895225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:18.956613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1026/2015] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:18.961568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.027390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32256c-2496-4e65-b48e-8f4533e2c9ed admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.031999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.032680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.032960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.033230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.033939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.035877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.066059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 990/2016] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:18 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:19.070602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.085052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1214d8-131c-4511-9e19-1e13781dedcc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.150071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.150809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.151262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.151541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.152248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.154203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.184739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1027/2017] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:19 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:19.188640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.199785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09d53-7dba-4b90-b11b-917b22c37b82 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.204767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.205359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.205704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.206041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.206881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.208722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.221733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 991/2018] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:19.226612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.243385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387e97-b37a-440c-8c8a-7129e739304a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.251914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.252808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.253083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.253349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] PATH_INFO: `/v3/projects/b77aec9f0a9b4738906155a971ae5a84/users/bbe9fdacce364bf1b616067dd01ea686/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.269386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77aec9f0a9b4738906155a971ae5a84, user_id=bbe9fdacce364bf1b616067dd01ea686, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.272657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.287550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1028/2019] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:19 2022] PUT /identity/v3/projects/b77aec9f0a9b4738906155a971ae5a84/users/bbe9fdacce364bf1b616067dd01ea686/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:19.291455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.302383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e00ce-8179-41f5-bf51-33dbf0c8871d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.316342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:19.373169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.373855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.374134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.374407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.375182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.376830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b537a6-70d8-4094-980a-ce60a2fece73 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.392349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 992/2020] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:19.394215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.406537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae6c4-9046-4b2f-89e2-80ad26410467 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.412380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.413309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.413589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.413862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] PATH_INFO: `/v3/projects/b77aec9f0a9b4738906155a971ae5a84/users/bbe9fdacce364bf1b616067dd01ea686/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.416009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77aec9f0a9b4738906155a971ae5a84, user_id=bbe9fdacce364bf1b616067dd01ea686, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.418717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.431853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1029/2021] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:19 2022] PUT /identity/v3/projects/b77aec9f0a9b4738906155a971ae5a84/users/bbe9fdacce364bf1b616067dd01ea686/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:19.442806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.443222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.443726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.481048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] MFA Rules not processed for user `bbe9fdacce364bf1b616067dd01ea686`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:19.482449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:19.565342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 993/2022] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:19.576500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:19.578312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:19.640409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b867-be53-472c-a245-da5825f276d5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:19.644113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:19.644775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:19.645050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:19.645311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:19.654330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:19.655801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71255cd-349d-48ab-9fb8-349d82b03baf service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:19.665829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1030/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:21.019160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:21.083775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605de01b-2c3f-49a3-bf78-e34d685cd979 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:21.088973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:21.089859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:21.090137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:21.090410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] PATH_INFO: `/v3/users/1b17b4c054a446ecb33c62bafa16cec5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:21.091669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b17b4c054a446ecb33c62bafa16cec5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:21.093585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:21.138047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:21.143617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b17b4c054a446ecb33c62bafa16cec5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.145216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b17b4c054a446ecb33c62bafa16cec5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.146050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b17b4c054a446ecb33c62bafa16cec5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.153150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b17b4c054a446ecb33c62bafa16cec5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.161942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b17b4c054a446ecb33c62bafa16cec5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.179103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 994/2024] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:21 2022] DELETE /identity/v3/users/1b17b4c054a446ecb33c62bafa16cec5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:21.183549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:21.184006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:21.184097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:21.202178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:21.203558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:21.209357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:21.266247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1031/2025] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:21.276525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:21.277965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:21.336097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee954-50de-4a42-aea9-9245fd511d43 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:21.339770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:21.340416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:21.340697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:21.340970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:21.350655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:21.352184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:21.362034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 995/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:21.683653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:21.685014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:21.698103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6ee0-ae87-4574-b573-8a6858fd2e9f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:21.703111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:21.703969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:21.704249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:21.704522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] PATH_INFO: `/v3/projects/183934503b5a405397a2d721701930c2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:21.706343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183934503b5a405397a2d721701930c2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:21.710529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:21.759158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183934503b5a405397a2d721701930c2', 'request_id': 'req-de6ab117-4a10-4e61-921a-c2a6f3681661'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.764644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183934503b5a405397a2d721701930c2', 'request_id': 'req-de6ab117-4a10-4e61-921a-c2a6f3681661'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.772010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183934503b5a405397a2d721701930c2', 'request_id': 'req-de6ab117-4a10-4e61-921a-c2a6f3681661'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.782867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] The token cache is being invalidate because project 183934503b5a405397a2d721701930c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:21.783149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.783763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.785796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1032/2027] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:21 2022] DELETE /identity/v3/projects/183934503b5a405397a2d721701930c2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:21.789715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:21.799227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4553c-e7ee-468d-acc6-f54082f87400 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:21.856227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:21.857092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:21.857374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:21.857646 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] PATH_INFO: `/v3/users/bbe9fdacce364bf1b616067dd01ea686` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:21.858849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe9fdacce364bf1b616067dd01ea686)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:21.860761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:21.905574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe9fdacce364bf1b616067dd01ea686'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.906228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe9fdacce364bf1b616067dd01ea686'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.906857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe9fdacce364bf1b616067dd01ea686'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.915123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe9fdacce364bf1b616067dd01ea686'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.922111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b86672eb-df62-4648-8c5e-28fe0755b677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe9fdacce364bf1b616067dd01ea686'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:21.923555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:21.942584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 996/2028] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:21 2022] DELETE /identity/v3/users/bbe9fdacce364bf1b616067dd01ea686 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:22.210136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.211746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:22.226216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ab117-4a10-4e61-921a-c2a6f3681661 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.287207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.288081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.288361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.288637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] PATH_INFO: `/v3/projects/b77aec9f0a9b4738906155a971ae5a84` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.289818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77aec9f0a9b4738906155a971ae5a84)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.291780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.333662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:22.343130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77aec9f0a9b4738906155a971ae5a84', 'request_id': 'req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:22.349187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77aec9f0a9b4738906155a971ae5a84', 'request_id': 'req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:22.357530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77aec9f0a9b4738906155a971ae5a84', 'request_id': 'req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:22.369583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] The token cache is being invalidate because project b77aec9f0a9b4738906155a971ae5a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:22.369868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:22.370521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-21bf4b2e-ce1f-4942-adbf-6bb5fe469af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:22.372080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1033/2029] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:22 2022] DELETE /identity/v3/projects/b77aec9f0a9b4738906155a971ae5a84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:22.376893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.377348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.377440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.395962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:22.397251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:22.460005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 997/2030] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:22.467616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.467906 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.468186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.488958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:22.490179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:22.499735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1034/2031] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:22.505912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.515917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e91af1-b62b-4abd-bade-d1e7938f8d35 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.521161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.521812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.522097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.522373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.523144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:22.523454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.525056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.536376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 998/2032] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:22.540715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.549996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff15ed-cf41-43db-adb5-9f50b2538729 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.554791 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.555457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.555722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.556111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.556928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.559376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.587126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1035/2033] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:22.591228 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.601313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b1fde-f906-42e3-b0f8-f7091f159f25 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.670638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.670846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.671110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.671523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.672239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.674162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.695792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 999/2034] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:22.700148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.712347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d10f3-20cb-492e-b4d4-6a9f7b8794a2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.716923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.717603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.717872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.718148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.718931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.720593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.732136 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1036/2035] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:22.736063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.746312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf43a2c-48e9-456b-9260-df67e62208ff admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.751088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.751749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.752029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.752303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.753036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.754816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228a9e9-553f-4335-9c47-d8189ca7f105 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.765390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1000/2036] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:22.769376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.780120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e84e4a-a23c-46a4-b8da-37a7b955b765 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:22.784765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:22.786195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.786473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.786748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] PATH_INFO: `/v3/projects/0f402394cc0844b69bb9943187eee694/users/a33b7916afed477187c46b21ac7ccf6a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.804741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f402394cc0844b69bb9943187eee694, user_id=a33b7916afed477187c46b21ac7ccf6a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:22.807456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:22.821120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1037/2037] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:22 2022] PUT /identity/v3/projects/0f402394cc0844b69bb9943187eee694/users/a33b7916afed477187c46b21ac7ccf6a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:22.831199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:22.831563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:22.831919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:22.859552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] MFA Rules not processed for user `a33b7916afed477187c46b21ac7ccf6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:22.860779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:22.894381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:22.928230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1001/2038] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:22.932680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:22.996823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5d1a7-6987-49d9-8f7b-f008e9d64f9e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.001475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.002120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.002399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.002666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.003385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.005271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.032179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1038/2039] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:22 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:23.036856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.047724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ba89-bb76-4827-80df-49abb82fd6fb None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.104824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.105494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.105776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.106049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.106747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.108737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.128858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1002/2040] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:23 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:23.132660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.142846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef70a25-9bcc-4537-8e53-24d815480878 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.147372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.148014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.148289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.148559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.149277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.150881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.162249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1039/2041] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:23.166340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.177043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7e673-2b6b-48d6-85c3-27dda289f3cb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.181891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.182853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.183132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.183404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] PATH_INFO: `/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.198894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e00a013922148fc9dd3b1b2a9fe54df, user_id=3c1ebbdf371a4cb8b9a8f1a9a8a34ceb, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.201558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.214175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1003/2042] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:23 2022] PUT /identity/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:23.218363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.230412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6c3c-2311-4ee9-98e4-1d70b6306ae0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.263878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:23.291249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.292048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.292475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.292926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.293842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.295604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23b15b-cce0-4f10-9f43-9cd0f2863543 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.306626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1040/2043] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:23.310679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.321789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca38e4-8608-40e0-8e8a-36612876c68f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.326702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.327632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.327920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.328240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] PATH_INFO: `/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.330218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e00a013922148fc9dd3b1b2a9fe54df, user_id=3c1ebbdf371a4cb8b9a8f1a9a8a34ceb, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.333550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.344071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1004/2044] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:23 2022] PUT /identity/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:23.354831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.355153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.355433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.381352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] MFA Rules not processed for user `3c1ebbdf371a4cb8b9a8f1a9a8a34ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:23.382608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:23.457615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1041/2045] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:23.469073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.470934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:23.528768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cf07b-6d77-449b-9f39-d74c8983d7ed admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.532398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.533032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.533310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.533581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.542284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.543730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392bc52-fa11-4b8c-b4b6-66f5fc508c31 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.553705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1005/2046] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:23.616851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.680169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba609ff-3226-47fb-9de4-ef1e6ab43afd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.685694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.688744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.688744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.688744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] PATH_INFO: `/v3/users/a33b7916afed477187c46b21ac7ccf6a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.688744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33b7916afed477187c46b21ac7ccf6a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.689912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.739453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33b7916afed477187c46b21ac7ccf6a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:23.740112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33b7916afed477187c46b21ac7ccf6a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:23.747277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33b7916afed477187c46b21ac7ccf6a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:23.751652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:23.755135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33b7916afed477187c46b21ac7ccf6a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:23.767697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33b7916afed477187c46b21ac7ccf6a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:23.774274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1042/2047] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:23 2022] DELETE /identity/v3/users/a33b7916afed477187c46b21ac7ccf6a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:23.778926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.779213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.779489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.796834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:23.798011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:23.824454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:23.862505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1006/2048] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:23.872825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:23.874426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:23.935467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9026c-5c27-47f6-8cfb-dee8d31e8c89 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:23.939677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:23.940417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:23.940701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:23.940885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:23.950732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:23.953207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:23.962388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1043/2049] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:24.254873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:24.256542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:24.273536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d788be3-10be-4f29-9a96-668e77f485c6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:24.279895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:24.281126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:24.281576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:24.282092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] PATH_INFO: `/v3/projects/0f402394cc0844b69bb9943187eee694` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:24.283318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f402394cc0844b69bb9943187eee694)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:24.285301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:24.336257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f402394cc0844b69bb9943187eee694', 'request_id': 'req-ede0caad-fd3e-4517-b601-769fee465776'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.343124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f402394cc0844b69bb9943187eee694', 'request_id': 'req-ede0caad-fd3e-4517-b601-769fee465776'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.351116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f402394cc0844b69bb9943187eee694', 'request_id': 'req-ede0caad-fd3e-4517-b601-769fee465776'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.361965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] The token cache is being invalidate because project 0f402394cc0844b69bb9943187eee694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:24.362252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.362900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.365132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1007/2050] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:24 2022] DELETE /identity/v3/projects/0f402394cc0844b69bb9943187eee694 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:24.369112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:24.379999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d1022-8a77-4d64-8a32-af3b091a5173 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:24.435926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:24.436792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:24.437067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:24.437336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] PATH_INFO: `/v3/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:24.438540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1ebbdf371a4cb8b9a8f1a9a8a34ceb)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:24.440489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:24.487851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1ebbdf371a4cb8b9a8f1a9a8a34ceb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.489056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1ebbdf371a4cb8b9a8f1a9a8a34ceb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.497232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1ebbdf371a4cb8b9a8f1a9a8a34ceb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.505359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1ebbdf371a4cb8b9a8f1a9a8a34ceb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.518970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b78a947f-2959-4590-8944-6f575f0b2708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1ebbdf371a4cb8b9a8f1a9a8a34ceb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.520503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:24.526689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1044/2051] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:24 2022] DELETE /identity/v3/users/3c1ebbdf371a4cb8b9a8f1a9a8a34ceb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:24.825821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:24.827367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:24.841531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0caad-fd3e-4517-b601-769fee465776 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:24.905096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:24.905959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:24.906236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:24.906506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] PATH_INFO: `/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:24.907710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e00a013922148fc9dd3b1b2a9fe54df)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:24.909602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:24.963378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e00a013922148fc9dd3b1b2a9fe54df', 'request_id': 'req-454523a3-59d6-476b-bf5f-310ed6eabc39'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.967855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:24.969014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e00a013922148fc9dd3b1b2a9fe54df', 'request_id': 'req-454523a3-59d6-476b-bf5f-310ed6eabc39'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.977244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e00a013922148fc9dd3b1b2a9fe54df', 'request_id': 'req-454523a3-59d6-476b-bf5f-310ed6eabc39'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.992137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] The token cache is being invalidate because project 2e00a013922148fc9dd3b1b2a9fe54df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:24.992137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.992137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-454523a3-59d6-476b-bf5f-310ed6eabc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:24.992492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1008/2052] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:24 2022] DELETE /identity/v3/projects/2e00a013922148fc9dd3b1b2a9fe54df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:24.997008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:24.997293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:24.997615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.015921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:25.017073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:25.080875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1045/2053] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:25.090349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.090642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.090956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.113209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:25.114497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:25.123942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1009/2054] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:25.128106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.140016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6804e-9763-4b6a-8499-86e04887320f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.147213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.147885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.148170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.148450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.149196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:25.149669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.151657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.166139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1046/2055] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:25.170059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.179872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2155bc-5c7d-4477-ab35-1017f8afe6c9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.185577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.186344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.186648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.187128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.188046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.190059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.223250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1010/2056] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:25.224637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.238893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be68d1-949c-4970-81a4-7006ee860809 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.298280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.298994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.299269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.299539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.300248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.302732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.334820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1047/2057] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:25.337288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.348340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abfc98-3f62-41a1-bb8a-793b5244a749 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.354363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.354363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.354363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.354654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.355323 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.356922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.370852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1011/2058] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:25.373456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.384100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4b132-5ab2-49ed-b476-c32dc8ac5a89 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.389004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.389680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.389962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.390236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.391039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.392790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa239-559b-49ee-8397-2c144c2ea9ef admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.404296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1048/2059] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:25.409223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.419752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c3df-95ea-4166-abc2-d1325a040c1f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.424567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.425450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.426344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.426619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] PATH_INFO: `/v3/projects/6f663385fbdb47dcb365080b2333b9ba/users/e30c6cf0c5f14f009a19d33a29dbfb7e/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.442012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f663385fbdb47dcb365080b2333b9ba, user_id=e30c6cf0c5f14f009a19d33a29dbfb7e, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.444847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.460498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1012/2060] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:25 2022] PUT /identity/v3/projects/6f663385fbdb47dcb365080b2333b9ba/users/e30c6cf0c5f14f009a19d33a29dbfb7e/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:25.469776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.470067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.470347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.497810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] MFA Rules not processed for user `e30c6cf0c5f14f009a19d33a29dbfb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:25.499074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:25.559009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:25.570446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1049/2061] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:25.574976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.636754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cc2fe-5865-4ee4-9deb-9690f52f8679 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.641463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.642113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.642394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.642665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.643416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.645869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.671032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1013/2062] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:25.676084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.687818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2537b20-56d6-4ba4-bd11-e8aa68733021 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.746531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.747299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.747689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.748109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.748913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.750919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.774296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1050/2063] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:25.778573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.788931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab917cd-1154-414e-a984-a38ba10aee24 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.793696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.794498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.794624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.794934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.795667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.797266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.808697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1014/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:25.812926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.824677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e99181-bed4-4ae2-9d26-22d36703b8a8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.829474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.830128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.830402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.830672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.831484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.833291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3acbe5-3566-44e9-b755-9dc54af6e2ae admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.844137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1051/2065] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:25.848288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:25.858956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1371a-8117-42de-b61b-c37b012e1ef0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:25.864340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:25.865232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.865513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.865787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] PATH_INFO: `/v3/projects/b28552d114f94dcfae1183fbcad1e5e4/users/101ce07849974bad8d8f834994206e12/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.881164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28552d114f94dcfae1183fbcad1e5e4, user_id=101ce07849974bad8d8f834994206e12, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:25.883869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:25.896497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1015/2066] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:25 2022] PUT /identity/v3/projects/b28552d114f94dcfae1183fbcad1e5e4/users/101ce07849974bad8d8f834994206e12/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:25.907022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:25.907611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:25.907611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:25.935925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] MFA Rules not processed for user `101ce07849974bad8d8f834994206e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:25.937127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.000710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1052/2067] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:26.005440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.040707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:26.071835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777c95d-6391-423f-bb39-2bbbdb7c07fa admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.076567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.077230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.077513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.077790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.078493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.080460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.106668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1016/2068] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:26 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:26.110650 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.121361 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b6b07-ed49-4896-9324-23453c9f3996 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.186629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.187559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.188504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.189213 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.190328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.193131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.205224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:26.228752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1053/2069] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:26 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:26.230082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.240848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef4f5-f0c9-4f0a-9915-2e1e48fba891 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.245999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.246698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.247011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.247290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.248031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.249817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.261394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1017/2070] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:26.265514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.276757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c284b-74df-40a4-9841-89635e737474 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.281538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.282570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.282887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.283196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] PATH_INFO: `/v3/projects/c556f64134f248fbb677aa5e95056a61/users/8a1e29c9429b43ed8899d949b17dffb9/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.298366 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c556f64134f248fbb677aa5e95056a61, user_id=8a1e29c9429b43ed8899d949b17dffb9, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.301137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.317077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1054/2071] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:26 2022] PUT /identity/v3/projects/c556f64134f248fbb677aa5e95056a61/users/8a1e29c9429b43ed8899d949b17dffb9/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:26.322110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.333255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776907e-6805-4316-9ffe-dbeedea9f3cc admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.392435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.393108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.393395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.393678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.394454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.396167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979997b-9e4c-4102-8f6a-83fa1a7854ab admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.406427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1018/2072] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:26.411065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.421408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406863dc-828b-439c-97e1-bf44ff4ce665 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.426751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.427743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.428020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.428289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] PATH_INFO: `/v3/projects/c556f64134f248fbb677aa5e95056a61/users/8a1e29c9429b43ed8899d949b17dffb9/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.430558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c556f64134f248fbb677aa5e95056a61, user_id=8a1e29c9429b43ed8899d949b17dffb9, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.433252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.444383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1055/2073] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:26 2022] PUT /identity/v3/projects/c556f64134f248fbb677aa5e95056a61/users/8a1e29c9429b43ed8899d949b17dffb9/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:26.455477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.455769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.456045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.485593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] MFA Rules not processed for user `8a1e29c9429b43ed8899d949b17dffb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:26.486835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.493318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.495847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.562599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1019/2074] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:26.569125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa73f9-22e0-4025-b6d0-ad6ea7dc0b11 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.574027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.574242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.575276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.575406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.575570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.575866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] PATH_INFO: `/v3/users/692115a495cd4af0891616e511db3491` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.577114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] RBAC: Authorizing `identity:delete_user(user_id=692115a495cd4af0891616e511db3491)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.579108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.608637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:26.629996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '692115a495cd4af0891616e511db3491'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:26.630663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '692115a495cd4af0891616e511db3491'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:26.639627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fc2c8-c339-4b16-899d-ecb3ac4adc93 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.643083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '692115a495cd4af0891616e511db3491'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:26.643912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.645178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.645463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.645733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.652610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '692115a495cd4af0891616e511db3491'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:26.658978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.661103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbfc8a-fa6a-4530-93ba-b29e17fe7420 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.665930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '692115a495cd4af0891616e511db3491'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:26.671502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1020/2075] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:26.673888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1056/2076] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:26 2022] DELETE /identity/v3/users/692115a495cd4af0891616e511db3491 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:26.678573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.678956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.679296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.698390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:26.699675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.715950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.717454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.755713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:26.766267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1021/2077] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:26.776566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:26.777898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.780471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a737609-8b34-4ad8-a48b-081a711bcacd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.784190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.784760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.785088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.785387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.787220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:26.837463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ad83d-ac21-42c7-a80a-1888cdd77536 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:26.841131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:26.841781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:26.842079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:26.842343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:26.855334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.857316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.857803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:26.859369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0fa70-5724-42a4-a454-1044178cc640 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:26.871502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1057/2078] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:26.874013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1022/2079] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:36:27.166025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.167509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:27.199520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b58f6-4b4b-4e14-a73f-037ff6ee52e3 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.204858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.205773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.206057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.206340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] PATH_INFO: `/v3/projects/569ae51482204d47835fe27d4a26e438` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.208303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569ae51482204d47835fe27d4a26e438)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.210235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.268178 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569ae51482204d47835fe27d4a26e438', 'request_id': 'req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:27.277592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569ae51482204d47835fe27d4a26e438', 'request_id': 'req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:27.285935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569ae51482204d47835fe27d4a26e438', 'request_id': 'req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:27.297525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] The token cache is being invalidate because project 569ae51482204d47835fe27d4a26e438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:27.297809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:27.298428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8a24f6bd-eef1-4418-bf5b-e5281bb755e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:27.299990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1058/2080] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:27 2022] DELETE /identity/v3/projects/569ae51482204d47835fe27d4a26e438 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:27.305154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.305445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.305730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.323662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:27.324857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:27.384027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1023/2081] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:27.392922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.393206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.393476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.412613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:27.413797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:27.421813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1059/2082] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:27.426089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.435632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c6946-630c-44b2-b16c-ed5d1328c3e6 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.440402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.440842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.441122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.441392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.442114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:27.442668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.444316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.453624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:27.461661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1024/2083] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:27.462507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.472271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2033734-5eb0-44f3-b67e-89efebae6188 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.479418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.479960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.482414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.482768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.483667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.485658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.518835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1060/2084] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:27 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:27.523891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.534863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab091968-8fad-4ba9-acd1-27fffc91d52d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.593428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.594028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.594312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.594590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.595361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.597286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.617872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1025/2085] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:27 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:36:27.621338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.632336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a25abc-f60c-4d94-b41d-cdc742a326bb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.636891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.637542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.637817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.638086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.638841 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.640437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.654259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1061/2086] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:27.655445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.665365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24790f2-cfb0-4532-b316-f0020bc7fb52 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.669928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.670598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.670914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.671183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.671896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.673479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0291d88-efb7-476a-a676-7f53e621302f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.683297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1026/2087] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:27.686991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.697516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e6d8a-1c62-413a-a8aa-93f9e03d334d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.702202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.703112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.703391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.703664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] PATH_INFO: `/v3/projects/e4e899ec13ff45fab993cb675ff684f9/users/a39c436205524c56aa0cf243e3ff53d4/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.718583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e899ec13ff45fab993cb675ff684f9, user_id=a39c436205524c56aa0cf243e3ff53d4, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.721314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.737206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1062/2088] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:27 2022] PUT /identity/v3/projects/e4e899ec13ff45fab993cb675ff684f9/users/a39c436205524c56aa0cf243e3ff53d4/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:27.747947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.748233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.748504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.774114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] MFA Rules not processed for user `a39c436205524c56aa0cf243e3ff53d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:27.775474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:27.844670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1027/2089] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:27.857734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:27.859453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:27.870122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:27.921183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ea6fb-ce3a-4ff1-b173-c8b7d2cd87e0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:27.924920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:27.926417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:27.926742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:27.927060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:27.935825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:27.937345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad035a5-4bcc-46fd-b995-4e9b399fbcf3 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:27.947576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1063/2090] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:29.754220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:29.755791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:29.824605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197712e-22d5-4ebe-bdb7-f88e82f6215c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:29.829650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:29.830528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:29.830839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:29.831117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] PATH_INFO: `/v3/users/e30c6cf0c5f14f009a19d33a29dbfb7e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:29.832322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30c6cf0c5f14f009a19d33a29dbfb7e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:29.834251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:29.845684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:29.891338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30c6cf0c5f14f009a19d33a29dbfb7e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:29.891974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30c6cf0c5f14f009a19d33a29dbfb7e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:29.892565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30c6cf0c5f14f009a19d33a29dbfb7e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:29.900090 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30c6cf0c5f14f009a19d33a29dbfb7e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:29.907354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30c6cf0c5f14f009a19d33a29dbfb7e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:29.924870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1028/2091] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:29 2022] DELETE /identity/v3/users/e30c6cf0c5f14f009a19d33a29dbfb7e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:29.930057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:29.930546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:29.930652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:29.947414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:29.948599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:30.008605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1064/2092] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:30.019092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:30.021035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:30.079932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084657b8-ef3c-4951-9a7b-55f341a2da4c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:30.083545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:30.084175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:30.084453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:30.084729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:30.093710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:30.095237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:30.105461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1029/2093] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:30.365027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:30.366590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:30.379975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6257-2fa6-4a4a-b7f7-2377640c557c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:30.384700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:30.386055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:30.386510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:30.386817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] PATH_INFO: `/v3/projects/6f663385fbdb47dcb365080b2333b9ba` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:30.387999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f663385fbdb47dcb365080b2333b9ba)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:30.389989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:30.436654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f663385fbdb47dcb365080b2333b9ba', 'request_id': 'req-cc20cd6e-8a1b-4000-a8db-7849f3489222'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.442883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f663385fbdb47dcb365080b2333b9ba', 'request_id': 'req-cc20cd6e-8a1b-4000-a8db-7849f3489222'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.450020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f663385fbdb47dcb365080b2333b9ba', 'request_id': 'req-cc20cd6e-8a1b-4000-a8db-7849f3489222'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.461623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] The token cache is being invalidate because project 6f663385fbdb47dcb365080b2333b9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:30.461924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.462920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.464206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1065/2094] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:30 2022] DELETE /identity/v3/projects/6f663385fbdb47dcb365080b2333b9ba => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:30.468304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:30.478722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f269eb-db27-48dd-8956-b694d4c4b4a5 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:30.535855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:30.536716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:30.536992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:30.537260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] PATH_INFO: `/v3/users/101ce07849974bad8d8f834994206e12` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:30.538425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101ce07849974bad8d8f834994206e12)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:30.540415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:30.575713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:30.590073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101ce07849974bad8d8f834994206e12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.590721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101ce07849974bad8d8f834994206e12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.591383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101ce07849974bad8d8f834994206e12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.598423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101ce07849974bad8d8f834994206e12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.605538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101ce07849974bad8d8f834994206e12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:30.622366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1030/2095] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:30 2022] DELETE /identity/v3/users/101ce07849974bad8d8f834994206e12 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:30.895513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:30.897018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:30.920020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20cd6e-8a1b-4000-a8db-7849f3489222 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:30.962355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:30.992193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:30.993210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:30.993317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:30.993574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] PATH_INFO: `/v3/projects/b28552d114f94dcfae1183fbcad1e5e4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:30.994811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28552d114f94dcfae1183fbcad1e5e4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:30.996701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.044388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28552d114f94dcfae1183fbcad1e5e4', 'request_id': 'req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.050654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28552d114f94dcfae1183fbcad1e5e4', 'request_id': 'req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.059101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28552d114f94dcfae1183fbcad1e5e4', 'request_id': 'req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.071013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] The token cache is being invalidate because project b28552d114f94dcfae1183fbcad1e5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:31.071392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.072154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.076744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1066/2096] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:30 2022] DELETE /identity/v3/projects/b28552d114f94dcfae1183fbcad1e5e4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:31.077978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.079459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:31.092743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484f93-f8ff-4746-a714-3016484051f0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:31.149741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:31.150597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.150906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.151181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] PATH_INFO: `/v3/users/8a1e29c9429b43ed8899d949b17dffb9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.152360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1e29c9429b43ed8899d949b17dffb9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:31.154276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.198810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1e29c9429b43ed8899d949b17dffb9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.199444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1e29c9429b43ed8899d949b17dffb9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.200050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1e29c9429b43ed8899d949b17dffb9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.207852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1e29c9429b43ed8899d949b17dffb9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.214625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-14ae457c-9ccf-4a0c-9c79-a8d579406460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1e29c9429b43ed8899d949b17dffb9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.231940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1031/2097] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:31 2022] DELETE /identity/v3/users/8a1e29c9429b43ed8899d949b17dffb9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:31.516492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.516492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:31.541910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc1b1-6eee-4e35-a8bd-c3c5dad2d393 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:31.601176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:31.601947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.602226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.602491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] PATH_INFO: `/v3/projects/c556f64134f248fbb677aa5e95056a61` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.603696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c556f64134f248fbb677aa5e95056a61)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:31.605586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.651895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c556f64134f248fbb677aa5e95056a61', 'request_id': 'req-537b27de-f9b2-4bad-9b32-1219e5048453'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.657364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c556f64134f248fbb677aa5e95056a61', 'request_id': 'req-537b27de-f9b2-4bad-9b32-1219e5048453'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.665212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c556f64134f248fbb677aa5e95056a61', 'request_id': 'req-537b27de-f9b2-4bad-9b32-1219e5048453'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.666354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:31.679375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] The token cache is being invalidate because project c556f64134f248fbb677aa5e95056a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:31.679665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.680289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-537b27de-f9b2-4bad-9b32-1219e5048453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:31.682065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1067/2098] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:31 2022] DELETE /identity/v3/projects/c556f64134f248fbb677aa5e95056a61 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:31.687672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.687961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.688245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.705799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:31.707542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:31.732838 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:31.768429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1032/2099] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:31.777372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.777657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.777928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.796159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:31.797334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:31.805099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1068/2100] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:31.809080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.818664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd72cfc-2b3a-4892-9845-7915f80f6eac None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:31.823244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:31.823881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.824161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.824434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.825152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:31.825458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:31.827089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.838944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1033/2101] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:31.843006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.852957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ee673-41e8-43d2-9bfb-8b57c8718ab9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:31.857525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:31.858158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.858435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.858703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.859421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:31.861308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.886349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1069/2102] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:31 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:31.890345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.900785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4ad90-1eb6-4239-b07a-fc55253f7b87 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:31.956038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:31.957194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:31.957473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:31.957743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:31.958435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:31.960402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:31.980207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1034/2103] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:31 2022] POST /identity/v3/users => generated 281 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:31.984949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:31.995616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594d39d-9b84-4054-bc1c-b0d030851429 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.005120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.005858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.006140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.006444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.007201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.008804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.019996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1070/2104] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:32.023949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.033842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571a11-f1ad-484d-ab19-d5c9d09df9a2 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.042702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.043464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.044410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.044691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.045418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.047194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736679-b04e-4899-82f0-fc31039900f8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.058396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1035/2105] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:32.062638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.073948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91355d-9bbd-494b-8cb1-f89852c20f67 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.078594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.079506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.079789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.080061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] PATH_INFO: `/v3/projects/7345be21135f4e73947093d6f134f406/users/ad72c32857974e139f41441c08b2061a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.094751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7345be21135f4e73947093d6f134f406, user_id=ad72c32857974e139f41441c08b2061a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.098068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.112394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1071/2106] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:32 2022] PUT /identity/v3/projects/7345be21135f4e73947093d6f134f406/users/ad72c32857974e139f41441c08b2061a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:32.125039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.125461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.125645 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.157057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] MFA Rules not processed for user `ad72c32857974e139f41441c08b2061a`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:32.158451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:32.230728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1036/2107] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:32.261092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.262805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:32.327292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd82f7e-b701-4b47-830d-fa60f9b15f78 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.330945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.331625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.332424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.332701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.341003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.342427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82267-96dd-4316-a07f-9c909a2041b1 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.351496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1072/2108] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:32.365420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.392186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:32.430909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496442ec-7be8-4b4a-89bc-ac27a2117e55 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.435521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.436386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.436667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.436941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] PATH_INFO: `/v3/users/ad72c32857974e139f41441c08b2061a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.438131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad72c32857974e139f41441c08b2061a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.440084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.485315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad72c32857974e139f41441c08b2061a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:32.485990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad72c32857974e139f41441c08b2061a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:32.486659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad72c32857974e139f41441c08b2061a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:32.498970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad72c32857974e139f41441c08b2061a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:32.506671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad72c32857974e139f41441c08b2061a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:32.525658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1037/2109] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:32 2022] DELETE /identity/v3/users/ad72c32857974e139f41441c08b2061a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:32.530341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.530634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.530993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.550000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:32.551904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:32.615334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1073/2110] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:32.624885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.626288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:32.684304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb77d6-a06c-4e9b-9065-56c5ba02e039 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:32.687920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:32.688550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:32.688826 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:32.689095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:32.698361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:32.699887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577d480-f37a-417e-b761-95a631d05d9e service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:32.709790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1038/2111] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:32.983385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:32.984789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:32.997300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798581e-06ee-4024-947b-b771ec70f275 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.001810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.002662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.002969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.003244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] PATH_INFO: `/v3/projects/7345be21135f4e73947093d6f134f406` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.005059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7345be21135f4e73947093d6f134f406)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.007013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.040028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:33.057852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7345be21135f4e73947093d6f134f406', 'request_id': 'req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:33.063416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7345be21135f4e73947093d6f134f406', 'request_id': 'req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:33.071182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7345be21135f4e73947093d6f134f406', 'request_id': 'req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:33.083221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] The token cache is being invalidate because project 7345be21135f4e73947093d6f134f406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:33.083517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:33.084337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7c5fa3d0-f415-41e3-9eea-16ad5c6f535a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:33.085844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1074/2112] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:32 2022] DELETE /identity/v3/projects/7345be21135f4e73947093d6f134f406 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:33.091225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.091576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.091895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.111450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:33.112907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:33.156288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:33.177191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1039/2113] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:33.184592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.185075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.185231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.204225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:33.205436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:33.214059 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1075/2114] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:33.218392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.228695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b67ac8-9869-4396-89e8-fd5fef20d6b8 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.233144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.233792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.234067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.234335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.235092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:33.235395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.237134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.249495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1040/2115] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:33.254123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.263467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1943b8-ff79-4c6c-8986-b4c4f160bd82 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.268146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.268795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.269104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.269385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.270078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.272037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.302351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1076/2116] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:33 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:33.303499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.313390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c6543-62f5-4e27-bc14-b1aad7d272da admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.374152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.374848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.375135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.375409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.376116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.378128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.402095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1041/2117] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:33 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:33.403313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.414019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0a47a-d319-4fb3-ae1c-436532f11f40 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.418758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.419436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.419712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.420119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.420849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.422441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.433420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1077/2118] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:36:33.437626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.448480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307522-12a1-4a9f-8bd6-a00ad200cc9e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.453138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.453782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.454062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.454335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.455109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.456740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c315a-5064-4990-a79d-10751aff6599 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.466881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1042/2119] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:33.470584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.480902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558401c-69fb-4868-8cd8-a5835fac27e7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.485577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.486457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.486739 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.487052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] PATH_INFO: `/v3/projects/1577716cf20f4274a95a51fb0fd6af99/users/0b78ec0ac9654ee39ae16ef205452e84/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.502247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1577716cf20f4274a95a51fb0fd6af99, user_id=0b78ec0ac9654ee39ae16ef205452e84, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.505523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.518278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1078/2120] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:33 2022] PUT /identity/v3/projects/1577716cf20f4274a95a51fb0fd6af99/users/0b78ec0ac9654ee39ae16ef205452e84/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:33.523906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e18d-ec40-42b4-a24c-afda24139c9c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.524197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e18d-ec40-42b4-a24c-afda24139c9c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.524472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e18d-ec40-42b4-a24c-afda24139c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.542445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3f73e18d-ec40-42b4-a24c-afda24139c9c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:33.543678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3f73e18d-ec40-42b4-a24c-afda24139c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:33.603851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1043/2121] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:33.613272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:33.615202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:33.681078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5165-496d-439e-b184-81f3bc05af8b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:33.684869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:33.685473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:33.685755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:33.686033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:33.696472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:33.697912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:33.708481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1079/2122] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:34.301311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cee62-2c97-4007-b14a-75e74d1dd393 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:34.301822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cee62-2c97-4007-b14a-75e74d1dd393 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:34.301932 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cee62-2c97-4007-b14a-75e74d1dd393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:34.329198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7f3cee62-2c97-4007-b14a-75e74d1dd393 None None] MFA Rules not processed for user `0b78ec0ac9654ee39ae16ef205452e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:34.330445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7f3cee62-2c97-4007-b14a-75e74d1dd393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:34.373614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:34.400630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1044/2123] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:34.421546 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:34.423102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:34.498097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37116be7-d68e-49c4-865f-4cbf0d78bb07 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:34.501707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:34.502370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:34.502646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:34.502966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:34.512287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:34.513722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:34.517510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:34.524370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1080/2124] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:37.991071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ecfc-83bb-4ce2-b322-47ed2cc46d3e None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:37.991563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ecfc-83bb-4ce2-b322-47ed2cc46d3e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:37.991777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ecfc-83bb-4ce2-b322-47ed2cc46d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:38.009845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-2e57ecfc-83bb-4ce2-b322-47ed2cc46d3e None None] MFA Rules not processed for user `0b78ec0ac9654ee39ae16ef205452e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:38.011089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2e57ecfc-83bb-4ce2-b322-47ed2cc46d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:38.018057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1045/2125] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:44.534381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:48.004710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=11 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:51.695726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:51.697167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:51.711399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d6881-05c4-4fec-81c9-624b15bf0a2f service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:51.716311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:51.717390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:51.717731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:51.718014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] PATH_INFO: `/v3/users/a39c436205524c56aa0cf243e3ff53d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:51.719529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39c436205524c56aa0cf243e3ff53d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:51.721837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:51.778365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39c436205524c56aa0cf243e3ff53d4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:51.779227 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39c436205524c56aa0cf243e3ff53d4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:51.787878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39c436205524c56aa0cf243e3ff53d4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:51.796838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39c436205524c56aa0cf243e3ff53d4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:51.819325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39c436205524c56aa0cf243e3ff53d4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:51.825775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1081/2126] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:51 2022] DELETE /identity/v3/users/a39c436205524c56aa0cf243e3ff53d4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:51.830014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:51.830449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:51.830623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:51.849889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:51.851139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:51.917570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1046/2127] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:51.927640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:51.929065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:51.987542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f89e8-b9f0-4974-976e-94cc775126df admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:51.991221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:51.991858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:51.992139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:51.992412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.001484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.003018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a27117-51fa-4125-9e84-8d3e433a99b8 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.014798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1082/2128] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:52.292114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.293519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:52.306677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b38ac3-899a-4066-8ff1-c0be743ccf56 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.312650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.313512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.313841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.314045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] PATH_INFO: `/v3/projects/e4e899ec13ff45fab993cb675ff684f9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.315330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e899ec13ff45fab993cb675ff684f9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.318685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.370939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e899ec13ff45fab993cb675ff684f9', 'request_id': 'req-688d8c26-03f9-4a0a-9696-65ac561c8964'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:52.376818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e899ec13ff45fab993cb675ff684f9', 'request_id': 'req-688d8c26-03f9-4a0a-9696-65ac561c8964'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:52.383698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e899ec13ff45fab993cb675ff684f9', 'request_id': 'req-688d8c26-03f9-4a0a-9696-65ac561c8964'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:52.394702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] The token cache is being invalidate because project e4e899ec13ff45fab993cb675ff684f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:52.395015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:52.395710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-688d8c26-03f9-4a0a-9696-65ac561c8964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:52.397533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1047/2129] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:52 2022] DELETE /identity/v3/projects/e4e899ec13ff45fab993cb675ff684f9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:52.402505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.402794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.403086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.421249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:52.422367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:52.485619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1083/2130] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:52.494159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.494159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.494409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.513684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:52.514943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:52.527074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1048/2131] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:52.528402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.549511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638e4a0-88fc-45d4-844b-8e39c309fdd6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.554946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.555656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.555938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.556212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.556974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:52.557279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.558896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.578672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1084/2132] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:52.579414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.590137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90ca9-1631-4527-8ef4-acadca23ecf1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.594883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.595542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.595825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.596100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.596810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.598847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.633107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1049/2133] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:52 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:52.634251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.641654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:52.653029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3f6e-e300-4536-87b9-8db23728fa8f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.721182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.721978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.722268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.723149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.723873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.726048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.746815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1085/2134] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:52 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:52.750601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.761032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239de16-8a3f-40db-876d-4a21b4784b54 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.765647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.766299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.766580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.766895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.767631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.769239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.780797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1050/2135] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:52.784405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.794856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198c03-affc-4712-ac74-8dcdeb13fede admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.799488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.800130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.800413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.800713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.801432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.803037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d400a-6c08-43a4-8ab8-15c9152113ba admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.813018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1086/2136] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:52.816615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.826798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1631d6-b955-4e49-b4d8-36735241b11a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:52.831819 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:52.833255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.833534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.833798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] PATH_INFO: `/v3/projects/1243c7bc4ea348f38e6bfe7a6f6aa8d5/users/c6766d41894c4936b4e317be82f03f0a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.849276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1243c7bc4ea348f38e6bfe7a6f6aa8d5, user_id=c6766d41894c4936b4e317be82f03f0a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:52.852372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:52.865168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1051/2137] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:52 2022] PUT /identity/v3/projects/1243c7bc4ea348f38e6bfe7a6f6aa8d5/users/c6766d41894c4936b4e317be82f03f0a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:36:52.874990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:52.875351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:52.875696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:52.902539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] MFA Rules not processed for user `c6766d41894c4936b4e317be82f03f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:52.903788 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:52.972390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1087/2138] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:52.984395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:52.986321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:53.008366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:53.051001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00724f8-7ddb-4660-9c88-d26908bdec55 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:53.055651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:53.056297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:53.056768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:53.057002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:53.067145 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:53.068612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f070062-f5da-40ed-9187-6d2727c5fb0a service glance] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:53.078339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1052/2139] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:53.101365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:53.102885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:53.222321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16bcd-34db-4ea0-83f1-3420afa46279 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:53.228597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:53.229488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:53.229770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:53.230042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] PATH_INFO: `/v3/users/2a06eee0b15b494b8c0e41b33fb75f07` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:53.231341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a06eee0b15b494b8c0e41b33fb75f07)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:53.234006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:53.251478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:53.304932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a06eee0b15b494b8c0e41b33fb75f07'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:53.305747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a06eee0b15b494b8c0e41b33fb75f07'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:53.317089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a06eee0b15b494b8c0e41b33fb75f07'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:53.326899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a06eee0b15b494b8c0e41b33fb75f07'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:53.342666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a06eee0b15b494b8c0e41b33fb75f07'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:53.350189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1088/2140] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:36:53 2022] DELETE /identity/v3/users/2a06eee0b15b494b8c0e41b33fb75f07 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:53.354714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:53.355075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:53.355390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:53.378190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:53.379486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:53.453053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1053/2141] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:53.460169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:53.461702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:53.529452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb200-da89-46e7-afef-9fd8017d91b9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:53.534575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:53.535819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:53.536126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:53.536396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:53.547992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:53.549478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57749ba-c6f9-447d-84bf-ffffc5728981 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:53.572803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1089/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:54.080066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.081351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.093553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d05ee-e6f0-4448-94a0-70f90e3b8a7a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.098743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.100068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.100589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.100905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] PATH_INFO: `/v3/projects/471779ee2c1f458ca81bfa5f042ecca6` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.102618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=471779ee2c1f458ca81bfa5f042ecca6)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.106117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.185518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471779ee2c1f458ca81bfa5f042ecca6', 'request_id': 'req-7d1e49a5-da11-4e86-bef7-c66162055c14'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:54.192300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471779ee2c1f458ca81bfa5f042ecca6', 'request_id': 'req-7d1e49a5-da11-4e86-bef7-c66162055c14'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:54.203093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471779ee2c1f458ca81bfa5f042ecca6', 'request_id': 'req-7d1e49a5-da11-4e86-bef7-c66162055c14'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:54.215489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] The token cache is being invalidate because project 471779ee2c1f458ca81bfa5f042ecca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:36:54.215866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:54.216561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7d1e49a5-da11-4e86-bef7-c66162055c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:36:54.218111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1054/2143] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:36:54 2022] DELETE /identity/v3/projects/471779ee2c1f458ca81bfa5f042ecca6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:54.224360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.225153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.225241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.244545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:54.245862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.310606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1090/2144] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:54.319020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.319552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.319990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.339300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:54.340518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.348792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1055/2145] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:54.354725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.365689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aecb0f-7b8d-448a-bf60-476073c61875 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.370601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.371297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.371582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.371860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.372594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:36:54.372908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.374503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.392499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1091/2146] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:36:54.394057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.403863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d95be4-4b82-4085-8e61-e4d1a77c7c8b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.408550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.409196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.409536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.409815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.410721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.412687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee67118-8aae-413e-acfb-59570d71148b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.445839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1056/2147] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:54.447189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.457990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442992e2-136d-46d5-b221-cc13ae28a1c2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.469020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.469634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.469924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.492199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] MFA Rules not processed for user `b1430d9ff64e4a9fa2f40bb31a098090`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:54.493492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.499462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:54.513283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:36:54.557161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.557842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.558279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.558549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.559284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.561287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.574816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1057/2148] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:36:54.590587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1092/2149] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:36:54.595291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.611236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170c834-5f7b-45ca-9f85-05cc3a9cdf2c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.616767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.617538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.617926 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.618300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.619183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.620916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.635126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1058/2150] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:54.640974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.655026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca284187-3bde-46e4-9739-d50d6346c7d6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.660822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.661490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.661892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.662170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.663071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.665124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5d738-4b7f-41b3-8516-0b0433211cd0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.709724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1093/2151] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:36:54.714757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.726768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd291e-03f2-49af-b6dd-01e905a1a2be admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.732556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.735639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.735965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.736233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] PATH_INFO: `/v3/projects/187f3726566d4c0bb0daf7b90cfe9a8b/users/5e8bcf1a91ba41e88d975817b51f0ee5/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.758870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=187f3726566d4c0bb0daf7b90cfe9a8b, user_id=5e8bcf1a91ba41e88d975817b51f0ee5, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.762161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.780617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1059/2152] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:36:54 2022] PUT /identity/v3/projects/187f3726566d4c0bb0daf7b90cfe9a8b/users/5e8bcf1a91ba41e88d975817b51f0ee5/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:36:54.785978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86508b6-9183-4486-9da0-62d9745a697e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.786337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86508b6-9183-4486-9da0-62d9745a697e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.786675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86508b6-9183-4486-9da0-62d9745a697e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.806704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c86508b6-9183-4486-9da0-62d9745a697e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:54.808031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c86508b6-9183-4486-9da0-62d9745a697e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.880357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1094/2153] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:54.891190 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:54.894086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:54.961429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502f48b-e968-429a-a741-c894340df0cf admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:54.965088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:54.965732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:54.967634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:54.967922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:54.980888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:54.982486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:54.992535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1060/2154] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:55.768521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f999f6-e6ef-4124-891a-1d1ac738ed3d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:55.769137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f999f6-e6ef-4124-891a-1d1ac738ed3d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:55.769137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f999f6-e6ef-4124-891a-1d1ac738ed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:55.796851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-10f999f6-e6ef-4124-891a-1d1ac738ed3d None None] MFA Rules not processed for user `5e8bcf1a91ba41e88d975817b51f0ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:55.798062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-10f999f6-e6ef-4124-891a-1d1ac738ed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:55.865143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1095/2155] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:36:55.875608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:36:55.877531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:55.938187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562afc4-3599-40b6-a2e4-eb8efb3645a1 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:36:55.942238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:36:55.942905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:55.943186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:55.943466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:55.956301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:36:55.958583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41491c8-22d6-4a01-b4d2-11aa927cdfe0 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:36:55.970276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1061/2156] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:36:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:36:57.348123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103faaa-0536-4405-ad7c-4b4fef41b69b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:36:57.348623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103faaa-0536-4405-ad7c-4b4fef41b69b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:36:57.348714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103faaa-0536-4405-ad7c-4b4fef41b69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:36:57.367142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8103faaa-0536-4405-ad7c-4b4fef41b69b None None] MFA Rules not processed for user `5e8bcf1a91ba41e88d975817b51f0ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:36:57.368328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8103faaa-0536-4405-ad7c-4b4fef41b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:36:57.375605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1096/2157] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:36:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:04.698398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:04.698894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:04.699148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:04.728880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] MFA Rules not processed for user `0b78ec0ac9654ee39ae16ef205452e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:04.730283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:04.749859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:04.840970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1062/2158] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:07.361704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:14.829874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=22 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:23.316042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e849-639d-46e0-886a-acd4f79ab63a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:23.316579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e849-639d-46e0-886a-acd4f79ab63a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:23.316847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e849-639d-46e0-886a-acd4f79ab63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:23.336594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0141e849-639d-46e0-886a-acd4f79ab63a None None] MFA Rules not processed for user `eceb9f042cb24343892f09454c7e685a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:23.337803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0141e849-639d-46e0-886a-acd4f79ab63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:23.404426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1097/2159] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:37:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:33.133491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:33.134980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:33.215748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66992d7-d4e9-4822-939a-1ae34ac49140 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:33.220460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:33.221340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:33.221615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:33.221887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] PATH_INFO: `/v3/users/c6766d41894c4936b4e317be82f03f0a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:33.223160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6766d41894c4936b4e317be82f03f0a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:33.227789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:33.285787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6766d41894c4936b4e317be82f03f0a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.286437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6766d41894c4936b4e317be82f03f0a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.287063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6766d41894c4936b4e317be82f03f0a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.295003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6766d41894c4936b4e317be82f03f0a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.302612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6766d41894c4936b4e317be82f03f0a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.323787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1063/2160] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:37:33 2022] DELETE /identity/v3/users/c6766d41894c4936b4e317be82f03f0a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:37:33.328274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:33.328821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:33.328821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:33.358317 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:33.359640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:33.433436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1098/2161] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:37:33.440242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:33.442889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:33.548990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc700d-88ec-4795-9ffe-d63f0d0cd1c8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:33.553306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:33.553998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:33.554223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:33.554555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:33.565476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:33.566984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0bae8-8962-4c41-8e07-1880d9dd507c service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:33.576803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1064/2162] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:37:33.843409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:33.844979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:33.857633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbefcd6-d1f2-4fc3-8829-5974e3fbc142 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:33.862552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:33.863919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:33.864382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:33.864851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] PATH_INFO: `/v3/projects/1243c7bc4ea348f38e6bfe7a6f6aa8d5` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:33.866666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1243c7bc4ea348f38e6bfe7a6f6aa8d5)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:33.869935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:33.926637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1243c7bc4ea348f38e6bfe7a6f6aa8d5', 'request_id': 'req-17b7a23c-8f18-42d3-97e6-b34d54fc8731'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.932034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1243c7bc4ea348f38e6bfe7a6f6aa8d5', 'request_id': 'req-17b7a23c-8f18-42d3-97e6-b34d54fc8731'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.940199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1243c7bc4ea348f38e6bfe7a6f6aa8d5', 'request_id': 'req-17b7a23c-8f18-42d3-97e6-b34d54fc8731'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.952437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] The token cache is being invalidate because project 1243c7bc4ea348f38e6bfe7a6f6aa8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:37:33.952719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.953385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:37:33.954950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1099/2163] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:37:33 2022] DELETE /identity/v3/projects/1243c7bc4ea348f38e6bfe7a6f6aa8d5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:37:37.784561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af853ff-02c4-45c3-a196-9c663ac6c600 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:37.785032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af853ff-02c4-45c3-a196-9c663ac6c600 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:37.785130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af853ff-02c4-45c3-a196-9c663ac6c600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:37.790141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:37.815993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8af853ff-02c4-45c3-a196-9c663ac6c600 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:37.817241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8af853ff-02c4-45c3-a196-9c663ac6c600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:37.889132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1065/2164] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:37:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:37.946798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:37.948209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:37.976635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:38.039262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7a23c-8f18-42d3-97e6-b34d54fc8731 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:38.044736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:38.046723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:38.047129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:38.047477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:38.056902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:38.058621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:38.069853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1100/2165] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:37:38.775987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:38.776352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:38.776695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:38.799113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:38.801489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:38.817380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1066/2166] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:37:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:38.829540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:38.841613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaf0ef-db41-4dab-8765-31c0b2f075d6 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:38.847005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:38.847657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:38.847943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:38.848853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:38.860336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:38.862011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b303b2-0f69-48a8-ba0b-bff59d42d4b4 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:38.871716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1101/2167] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:37:48.792818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=33 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:48.868480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=28 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:54.622862 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:54.624994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:54.741920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65be56-5f87-48af-b3b7-8116282908aa None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:54.748952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:54.749714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:54.750206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:54.750629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:54.753159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:54.769093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:54.771397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243e75b-457a-4805-ba80-19632c97b8a8 service placement] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:54.785541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1067/2168] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:37:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:37:56.347556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.348139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.348139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.350709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1102/2169] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:37:56.357168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a6d-0c5a-4a74-9fd5-8cd731fdeb5e None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.357683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a6d-0c5a-4a74-9fd5-8cd731fdeb5e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.358258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8a6d-0c5a-4a74-9fd5-8cd731fdeb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.430897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b84e8a6d-0c5a-4a74-9fd5-8cd731fdeb5e None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:56.432441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b84e8a6d-0c5a-4a74-9fd5-8cd731fdeb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:56.528677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1068/2170] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:37:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:56.547449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:56.549167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:56.633032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2ea7-ba54-4945-9e5b-1b3f573ec06a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:56.637034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:56.637681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.637963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.638238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.649398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:56.650866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573573-e328-4a3e-9579-a873d14a1197 service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:56.663487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1103/2171] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:37:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:37:56.760655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.760951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.761398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.762890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1069/2172] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:37:56.770029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599fa6-8b5f-4289-b376-d4ed6d980fee None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.771207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599fa6-8b5f-4289-b376-d4ed6d980fee None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.771584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599fa6-8b5f-4289-b376-d4ed6d980fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.793092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0b599fa6-8b5f-4289-b376-d4ed6d980fee None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:56.794334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0b599fa6-8b5f-4289-b376-d4ed6d980fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:56.866836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1104/2173] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:37:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:56.882592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:56.884354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:56.900617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac13e1-8034-4291-b849-c74277222a0f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:56.904336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:56.905002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:56.905278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:56.905545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:56.914257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:56.915753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b7982-983d-4f9a-81c0-41faeaf2681c service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:56.924957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1070/2174] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:37:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:37:57.036935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:57.037228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:57.037509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:57.042570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1105/2175] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:37:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:37:57.045487 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91219e4d-d98d-46f2-ac62-435be9d2c588 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:57.045774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91219e4d-d98d-46f2-ac62-435be9d2c588 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:57.046049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91219e4d-d98d-46f2-ac62-435be9d2c588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:57.061790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:37:57.068125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-91219e4d-d98d-46f2-ac62-435be9d2c588 None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:57.069348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-91219e4d-d98d-46f2-ac62-435be9d2c588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:57.084713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1071/2176] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:37:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:57.098246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:57.100855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:57.117446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971d04-6924-40f2-8ff9-9321357c7e51 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:57.121431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:57.122319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:57.122603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:57.122921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:57.139597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:57.141199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:57.157344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1106/2177] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:37:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:37:59.212382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:59.212834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:59.212938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:59.261457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:37:59.262654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:37:59.279009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1072/2178] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:37:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:37:59.291938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:37:59.313208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff00a3f-b214-4130-9c00-49c10d797026 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:37:59.318803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:37:59.319440 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:37:59.319723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:37:59.320000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:37:59.328054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:37:59.329632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:37:59.338398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1107/2179] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:37:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 00:38:01.355534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:01.357079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:01.432149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d806-0c74-4017-b8bd-0dce96cb06b3 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:01.436635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:01.437495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:01.437776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:01.438047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] PATH_INFO: `/v3/users/0b78ec0ac9654ee39ae16ef205452e84` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:01.439307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b78ec0ac9654ee39ae16ef205452e84)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:01.441221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:01.486560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b78ec0ac9654ee39ae16ef205452e84'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.487235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b78ec0ac9654ee39ae16ef205452e84'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.487859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b78ec0ac9654ee39ae16ef205452e84'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.498803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b78ec0ac9654ee39ae16ef205452e84'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.505923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9a3e146b-15d4-416b-b071-e01ea7e66d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b78ec0ac9654ee39ae16ef205452e84'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.523054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1073/2180] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:38:01 2022] DELETE /identity/v3/users/0b78ec0ac9654ee39ae16ef205452e84 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:01.696050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:01.707648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30314eb9-db61-4ec6-9922-dc0de749056c service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:01.761466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:01.767703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:01.768560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:01.768846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:01.769122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] PATH_INFO: `/v3/projects/1577716cf20f4274a95a51fb0fd6af99` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:01.770331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1577716cf20f4274a95a51fb0fd6af99)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:01.772274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:01.818832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1577716cf20f4274a95a51fb0fd6af99', 'request_id': 'req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.824800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1577716cf20f4274a95a51fb0fd6af99', 'request_id': 'req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.833702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1577716cf20f4274a95a51fb0fd6af99', 'request_id': 'req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.845353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] The token cache is being invalidate because project 1577716cf20f4274a95a51fb0fd6af99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:38:01.845638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.846299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-211c61c7-9ad0-47e3-aeec-3b0e09b92af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:01.847880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1108/2181] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:38:01 2022] DELETE /identity/v3/projects/1577716cf20f4274a95a51fb0fd6af99 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:01.853739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:01.854029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:01.854304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:01.890065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:01.891285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:01.909658 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:01.951915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1074/2182] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:01.959858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:01.960143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:01.960417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:01.979273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:01.980453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:01.988321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1109/2183] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:01.992453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.001928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d8015-eaeb-4016-b957-5b75ac0a8251 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.006962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.007594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.007972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.008246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.008991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:38:02.009294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.010901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.022839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1075/2184] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:38:02.026975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.036034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0790-3cdb-40c8-bfe5-46e70d686f39 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.040665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.041292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.041572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.041842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.042532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.044499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.069527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1110/2185] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:38:02 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:38:02.073405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.083781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe324-bf28-4995-b0a5-7c0fa09dd24d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.139342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.140024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.140306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.140581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.141264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.143218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.162753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1076/2186] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:38:02 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:38:02.166898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.177793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf165d-9786-43d9-9679-9d55873cfefe admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.182457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.183142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.183424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.183696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.184424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.186050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.196458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1111/2187] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:38:02.200264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.210249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c48e6-fad1-4692-a84a-cab1c2bdf136 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.214927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.215569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.215848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.216123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.216845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.218430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85b95-9e7a-412c-ae22-791265ab00cf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.228954 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1077/2188] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:38:02.232871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.243542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722f850-2c9d-422d-87e0-43a4c4104a6b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.248223 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.249094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.249457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.249747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] PATH_INFO: `/v3/projects/ee101f4ad5114947adca2752e1297b1d/users/cacab6cab38e4394bfbe1adc841feaa9/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.264119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee101f4ad5114947adca2752e1297b1d, user_id=cacab6cab38e4394bfbe1adc841feaa9, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.266766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.279115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1112/2189] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:38:02 2022] PUT /identity/v3/projects/ee101f4ad5114947adca2752e1297b1d/users/cacab6cab38e4394bfbe1adc841feaa9/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:02.289746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6397-6946-40c0-ac6b-ebb65158dcbe None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.290038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6397-6946-40c0-ac6b-ebb65158dcbe None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.290318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6397-6946-40c0-ac6b-ebb65158dcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.316484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3e2b6397-6946-40c0-ac6b-ebb65158dcbe None None] MFA Rules not processed for user `cacab6cab38e4394bfbe1adc841feaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:02.317648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3e2b6397-6946-40c0-ac6b-ebb65158dcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:02.381329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1078/2190] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:02.392498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:02.394144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:02.492365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d60e-d49f-44c4-9684-329afb6578a9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:02.497958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:02.498611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:02.498929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:02.499203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:02.508997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:02.510429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:02.520040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1113/2191] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:38:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:38:10.382707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:10.383114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:10.383114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:10.402485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:10.403721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:10.419348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:10.479625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1079/2192] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:38:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:10.488324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:10.495481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:10.505732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:10.593868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97c269-2434-424e-a895-20468ebde54e service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:10.599780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:10.600430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:10.600713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:10.600989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:10.610264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:10.611379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f020433-9f50-4328-b291-ed7de64b4a26 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:10.620044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1114/2193] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:38:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:38:15.481543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:15.563265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d099147-8359-49ef-aa70-56ea8527a4e5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:15.571757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:15.572696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:15.572994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:15.573271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] PATH_INFO: `/v3/users/cacab6cab38e4394bfbe1adc841feaa9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:15.574514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] RBAC: Authorizing `identity:delete_user(user_id=cacab6cab38e4394bfbe1adc841feaa9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:15.576554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:15.635779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cacab6cab38e4394bfbe1adc841feaa9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:15.636488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cacab6cab38e4394bfbe1adc841feaa9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:15.637080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cacab6cab38e4394bfbe1adc841feaa9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:15.645746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cacab6cab38e4394bfbe1adc841feaa9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:15.654741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cacab6cab38e4394bfbe1adc841feaa9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:15.672812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1080/2194] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:38:15 2022] DELETE /identity/v3/users/cacab6cab38e4394bfbe1adc841feaa9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:15.677376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:15.677956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:15.677956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:15.696274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:15.697487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:15.759399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1115/2195] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:15.769087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:15.770585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:15.818727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:15.831470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fd6f-1cf5-445d-92ac-d7005f1bfade admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:15.836229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:15.836869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:15.837146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:15.837418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:15.848149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:15.851602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f421002-e2b2-4e4a-a177-b7fc35019806 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:15.872953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1081/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:38:16.159640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.161073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:16.174357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8767c-49f1-4e23-9f17-cf98ffaf141e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.179694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.180571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.180856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.181134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] PATH_INFO: `/v3/projects/ee101f4ad5114947adca2752e1297b1d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.182354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee101f4ad5114947adca2752e1297b1d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.184320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.240275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee101f4ad5114947adca2752e1297b1d', 'request_id': 'req-69d569de-a315-4dcd-afdf-de42e9995769'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:16.246692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee101f4ad5114947adca2752e1297b1d', 'request_id': 'req-69d569de-a315-4dcd-afdf-de42e9995769'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:16.255160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee101f4ad5114947adca2752e1297b1d', 'request_id': 'req-69d569de-a315-4dcd-afdf-de42e9995769'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:16.266717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] The token cache is being invalidate because project ee101f4ad5114947adca2752e1297b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:38:16.267033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:16.267724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-69d569de-a315-4dcd-afdf-de42e9995769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:16.269263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1116/2197] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:38:16 2022] DELETE /identity/v3/projects/ee101f4ad5114947adca2752e1297b1d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:16.274207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.274496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.274797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.293279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:16.294499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:16.360817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1082/2198] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:16.369992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.370376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.370661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.390477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:16.391903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:16.405802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1117/2199] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:16.407088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.417307 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b2cb-e0e3-462a-a1a6-f4be5212bfe5 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.421922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.422583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.422892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.423165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.423898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:38:16.424209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.425969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.438348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1083/2200] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:38:16.443245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.447681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:16.454614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313887-39cd-4dcc-ac2f-99e0cc5eb8e3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.461629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.462297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.462580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.462895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.463633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.465555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.498296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1118/2201] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:38:16 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:38:16.499486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.509808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867951e-7684-4572-917d-5c1ec57a4100 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.569677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.570769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.571086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.571361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.572156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.574297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.596739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1084/2202] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:38:16 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:38:16.607027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.627106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99a1c-f9d9-4a1b-926c-1c873e615042 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.633677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.635156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.635544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.635924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.636772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.638636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.653444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1119/2203] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:38:16.657437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.680151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16074554-91ce-4da2-8d12-ba8e82bc8bba admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.685799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.686594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.686996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.687277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.688117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.689755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4781c54-d007-4ce0-b33f-752ffcb0b1d0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.701164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1085/2204] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:38:16.705250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.735591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162589a-2bb7-4452-8e34-3ca934258953 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:16.741491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:16.742509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.742933 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.743281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] PATH_INFO: `/v3/projects/8b93d82c53924aa19d3c7f9bb6cc2074/users/fc285db17b4f4453afe2a4dd8af1378f/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.760463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b93d82c53924aa19d3c7f9bb6cc2074, user_id=fc285db17b4f4453afe2a4dd8af1378f, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:16.763663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:16.779172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1120/2205] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:38:16 2022] PUT /identity/v3/projects/8b93d82c53924aa19d3c7f9bb6cc2074/users/fc285db17b4f4453afe2a4dd8af1378f/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:16.848498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:16.849037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:16.849312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:16.878690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] MFA Rules not processed for user `fc285db17b4f4453afe2a4dd8af1378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:16.879979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:16.936721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:16.970205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1086/2206] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:16.982508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:16.984196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:17.045446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8b1c2-b8a5-4291-ace7-c75eb9762e9b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:17.049201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:17.049873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:17.050157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:17.050424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:17.060399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:17.061916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a27c4e-4f56-4764-88f9-662fe3cc2c63 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:17.071509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1121/2207] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:38:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:38:26.253872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:26.327036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104285f-e70a-411a-b757-6b7d8131f117 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:26.331453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:26.332317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:26.332593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:26.332865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] PATH_INFO: `/v3/users/fc285db17b4f4453afe2a4dd8af1378f` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:26.334031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc285db17b4f4453afe2a4dd8af1378f)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:26.336472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:26.381807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc285db17b4f4453afe2a4dd8af1378f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.382508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc285db17b4f4453afe2a4dd8af1378f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.383135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc285db17b4f4453afe2a4dd8af1378f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.390322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc285db17b4f4453afe2a4dd8af1378f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.397799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc285db17b4f4453afe2a4dd8af1378f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.415173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1087/2208] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:38:26 2022] DELETE /identity/v3/users/fc285db17b4f4453afe2a4dd8af1378f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:26.419721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:26.420222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:26.420335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:26.436738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:26.437912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:26.501478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1122/2209] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:26.511048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:26.512424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:26.570795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221e30f-dfea-4e09-bd52-b5db3ee9d389 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:26.574385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:26.575041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:26.575319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:26.575587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:26.584429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:26.585895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7564e2-76a1-402c-ad99-e8bdab1391c2 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:26.595130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1088/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:38:26.859294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:26.860786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:26.873453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a58c0-d04c-49d2-b613-d98f6c288121 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:26.878183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:26.879106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:26.879374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:26.879646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] PATH_INFO: `/v3/projects/8b93d82c53924aa19d3c7f9bb6cc2074` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:26.880846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b93d82c53924aa19d3c7f9bb6cc2074)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:26.882770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:26.925968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:26.930740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b93d82c53924aa19d3c7f9bb6cc2074', 'request_id': 'req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.936825 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b93d82c53924aa19d3c7f9bb6cc2074', 'request_id': 'req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.945677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b93d82c53924aa19d3c7f9bb6cc2074', 'request_id': 'req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.956658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] The token cache is being invalidate because project 8b93d82c53924aa19d3c7f9bb6cc2074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:38:26.956940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.957559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:38:26.959119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1123/2211] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:38:26 2022] DELETE /identity/v3/projects/8b93d82c53924aa19d3c7f9bb6cc2074 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:38:36.591005 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:36.953592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:37.389096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3564b-1731-40ee-ac9d-5fdadec62087 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:37.389461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3564b-1731-40ee-ac9d-5fdadec62087 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:37.389741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3564b-1731-40ee-ac9d-5fdadec62087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:37.408052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-54a3564b-1731-40ee-ac9d-5fdadec62087 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:37.409243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-54a3564b-1731-40ee-ac9d-5fdadec62087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:37.467293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1089/2212] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:38:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:37.490179 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:38:37.491795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:37.562738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3e23d-c50e-4be1-9fbd-5b1e0b640127 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:38:37.569463 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:38:37.570110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:37.570391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:37.570666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:37.579102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:38:37.580690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127114c1-bdca-4393-a048-539d11bbf75a service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:38:37.588832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1124/2213] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:38:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:38:47.460552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=25 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:47.585439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=32 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:38:47.912419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff307-8989-4560-b58a-ce498ea5ad08 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:47.912707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff307-8989-4560-b58a-ce498ea5ad08 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:47.912989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff307-8989-4560-b58a-ce498ea5ad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:47.932368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e19ff307-8989-4560-b58a-ce498ea5ad08 None None] MFA Rules not processed for user `5e8bcf1a91ba41e88d975817b51f0ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:47.933647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e19ff307-8989-4560-b58a-ce498ea5ad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:47.989333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1090/2214] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:57.850189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae168-03c5-4777-89d7-b1d7e8bbcdbe None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:38:57.850851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae168-03c5-4777-89d7-b1d7e8bbcdbe None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:38:57.850851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae168-03c5-4777-89d7-b1d7e8bbcdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:38:57.867988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-231ae168-03c5-4777-89d7-b1d7e8bbcdbe None None] MFA Rules not processed for user `b1430d9ff64e4a9fa2f40bb31a098090`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:38:57.869172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-231ae168-03c5-4777-89d7-b1d7e8bbcdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:38:57.922545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1125/2215] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:38:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:38:57.982504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=25 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:06.439485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:06.440113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:06.440113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:06.459050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] MFA Rules not processed for user `eceb9f042cb24343892f09454c7e685a`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:39:06.460362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:06.518257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1091/2216] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:39:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:39:07.916565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=25 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:16.511134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=25 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:32.990407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:32.990964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:32.990964 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:33.008091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] MFA Rules not processed for user `dc56e0e37b3d4445a69391142935abcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:39:33.009265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:33.062487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1126/2217] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:39:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:39:35.879629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:35.881146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:35.937382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a81f-aac4-4a48-b9e1-4f4fe483adf4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:35.940947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:35.941612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:35.941894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:35.942169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:35.943957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:36.022035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:36.023511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:36.036159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1092/2218] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:39:37.146075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:37.147856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:37.161477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1ce8d-cbca-4c55-9670-045dc6f4f913 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:37.165164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:37.165860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:37.166141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:37.166416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:37.168211 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:37.181522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:37.183009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:37.195506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1127/2219] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:39:46.030682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=54 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:47.189937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=31 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:47.343066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:47.352287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d42e-027e-4c6c-b69e-bd2d10eb27b8 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:47.356944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:47.357859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:47.358146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:47.358446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:47.360567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:47.373648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:47.375272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:47.384398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1093/2220] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:39:56.115975 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:56.117939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.181647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:56.183175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.202885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a747e-62e0-4026-8882-4bf70284170d service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:56.208309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:56.209118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.209611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.209730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.211699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.225824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:56.227444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:56.238312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1128/2221] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:39:56.256384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8efd1-16d9-474d-b5a1-e78bc88c62da service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:56.261016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:56.261880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.262162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.262436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] PATH_INFO: `/v3/users/5e8bcf1a91ba41e88d975817b51f0ee5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.264000 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8bcf1a91ba41e88d975817b51f0ee5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:56.265607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:56.311898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8bcf1a91ba41e88d975817b51f0ee5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.312518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8bcf1a91ba41e88d975817b51f0ee5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.313107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8bcf1a91ba41e88d975817b51f0ee5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.320281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8bcf1a91ba41e88d975817b51f0ee5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.327152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8bcf1a91ba41e88d975817b51f0ee5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.344428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1094/2222] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:39:56 2022] DELETE /identity/v3/users/5e8bcf1a91ba41e88d975817b51f0ee5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:39:56.708169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:56.719887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752b77-35d8-43d0-b7cb-e2c1d753a7e8 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:56.751765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:56.753529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.783288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:56.784294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.784698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.785069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] PATH_INFO: `/v3/projects/187f3726566d4c0bb0daf7b90cfe9a8b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.786510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187f3726566d4c0bb0daf7b90cfe9a8b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:56.788562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:56.834299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:56.838929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187f3726566d4c0bb0daf7b90cfe9a8b', 'request_id': 'req-a851080a-10fa-4ae9-b83d-91310bb7aea0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.840830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0708639-cc24-4456-a2a2-24c074cadb0c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:56.844564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187f3726566d4c0bb0daf7b90cfe9a8b', 'request_id': 'req-a851080a-10fa-4ae9-b83d-91310bb7aea0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.853006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187f3726566d4c0bb0daf7b90cfe9a8b', 'request_id': 'req-a851080a-10fa-4ae9-b83d-91310bb7aea0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.864569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] The token cache is being invalidate because project 187f3726566d4c0bb0daf7b90cfe9a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:39:56.864854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.865487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a851080a-10fa-4ae9-b83d-91310bb7aea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:39:56.867070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1129/2223] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:39:56 2022] DELETE /identity/v3/projects/187f3726566d4c0bb0daf7b90cfe9a8b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:39:56.872524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899550-7338-463b-bd04-7f6bc35c6cb8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.872802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899550-7338-463b-bd04-7f6bc35c6cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.873093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899550-7338-463b-bd04-7f6bc35c6cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.891237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-19899550-7338-463b-bd04-7f6bc35c6cb8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:39:56.892417 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-19899550-7338-463b-bd04-7f6bc35c6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.906171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:56.913834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:56.914516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.914824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.915105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.916963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.956037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1130/2224] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:39:56.963469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:56.963762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:56.964044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:56.975522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:56.977173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:56.981790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:39:56.983010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:56.986156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1095/2225] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:39:56.990887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1131/2226] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:39:56.995472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.005606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefafb9-cb09-4f8b-a4e0-a7eec2125b21 service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.010922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.011566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.011842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.012229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.012958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:39:57.013258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.014850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.026243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1096/2227] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:39:57.030487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.039839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46c423-3b6f-4e9f-be5a-a98b406c76fb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.044563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.045221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.045500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.045773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.046472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.048427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.076228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1132/2228] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:39:57 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:39:57.080546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.090611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c1b12-9553-4d21-acb5-8848942e8311 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.148263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.148928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.149211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.149489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.150757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.152689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.175247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1097/2229] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:39:57 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:39:57.179295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.189541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb83cb-a751-42e6-88bf-416e6f8a03fc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.194198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.194886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.195170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.195441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.196156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.197741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.208365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1133/2230] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:39:57.212162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.223979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2aa3b0-4914-4f23-9609-0c9e57cb27d5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.229529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.230198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.230482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.230758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.231558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.233183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453eef0-4d1f-4954-8ce1-88b4ee35dc07 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.246438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1098/2231] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:39:57.250521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.261728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaf012-2b4e-4f22-a6d4-c2c436b3aaa2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.266306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.267195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.267471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.267744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] PATH_INFO: `/v3/projects/10b82c1cb87645d9831a2635a2aa01df/users/9dc07d7e2ff644c0942c2aaf5b498656/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.282229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b82c1cb87645d9831a2635a2aa01df, user_id=9dc07d7e2ff644c0942c2aaf5b498656, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.284894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.297943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1134/2232] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:39:57 2022] PUT /identity/v3/projects/10b82c1cb87645d9831a2635a2aa01df/users/9dc07d7e2ff644c0942c2aaf5b498656/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:39:57.303207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.303493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.303768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.316016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:39:57.322585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:39:57.324528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:57.381983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1099/2233] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:39:57.391883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:57.393538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:57.449848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93eb4a-3513-4e16-ab8f-a154984ddf1a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:57.453408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:57.454042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:57.454318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:57.454589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:57.464135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:57.465631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cca09-ca2c-470f-8798-489783fbc436 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:57.475268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1135/2234] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:39:59.256166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:39:59.258659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:59.376640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5f568-b409-42a4-a6ef-38629bad84fd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:39:59.383744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:39:59.385143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:39:59.385586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:39:59.385957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:39:59.388147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:39:59.404179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:39:59.408924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:39:59.432261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1100/2235] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:39:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:02.271884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.272419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.272419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.299960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] MFA Rules not processed for user `9dc07d7e2ff644c0942c2aaf5b498656`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:02.300970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:02.358810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:02.377663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1136/2236] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:02.382040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.392503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057527d8-b6a4-4dec-b17e-a1cef8f14c90 service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.397100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.397738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.398018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.398291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.399030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.401465 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.427568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1101/2237] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:02 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:02.431639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.442275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a0ceb-6ab5-4544-afe9-b946cbdb5c38 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.499034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.499790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.500074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.500388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.501098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.503090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.523145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1137/2238] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:02 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:02.527757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.538185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa64a4-27d3-4464-bee0-d24c4eb2e996 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.542802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.543459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.543734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.544002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.544723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.546299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.556706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1102/2239] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:02.560789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.571687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c574-14b0-4a12-9214-39e79f78a149 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.577169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.578135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.578545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.578974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] PATH_INFO: `/v3/projects/ab51d5751c294b7ba2f5faddce141e3e/users/21a37f2be5944f8c8d7f4e11b0eff395/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.595949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab51d5751c294b7ba2f5faddce141e3e, user_id=21a37f2be5944f8c8d7f4e11b0eff395, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.598896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.612388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1138/2240] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:02 2022] PUT /identity/v3/projects/ab51d5751c294b7ba2f5faddce141e3e/users/21a37f2be5944f8c8d7f4e11b0eff395/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:02.616539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.626700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4566d1-9948-4fcd-bb92-b510bc34e8df admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.682634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.683160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.683474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.683751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.684476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.686071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60652a1-dde8-448d-96a3-703fe763ba56 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.688062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:02.699003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1103/2241] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:02.702919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:02.713836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030a61-ecff-434f-81e3-21aaf7cb6f4c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:02.718646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:02.719555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:02.719834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:02.720102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] PATH_INFO: `/v3/projects/ab51d5751c294b7ba2f5faddce141e3e/users/21a37f2be5944f8c8d7f4e11b0eff395/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:02.722075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab51d5751c294b7ba2f5faddce141e3e, user_id=21a37f2be5944f8c8d7f4e11b0eff395, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:02.725007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:02.735667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1139/2242] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:02 2022] PUT /identity/v3/projects/ab51d5751c294b7ba2f5faddce141e3e/users/21a37f2be5944f8c8d7f4e11b0eff395/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:07.284327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b962-5bce-4554-9154-3d410f6ee4b4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:07.284327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b962-5bce-4554-9154-3d410f6ee4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:07.284902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b962-5bce-4554-9154-3d410f6ee4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:07.316151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5571b962-5bce-4554-9154-3d410f6ee4b4 None None] MFA Rules not processed for user `21a37f2be5944f8c8d7f4e11b0eff395`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:07.317382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5571b962-5bce-4554-9154-3d410f6ee4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:07.406922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1104/2243] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:07.418011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:07.420909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:07.503302 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2182fc8-ce6a-4d73-9de0-3de30e4e9972 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:07.507877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:07.509003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:07.511032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:07.511406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:07.613515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:07.617318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:07.619866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:07.630065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1140/2244] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:07.749272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:07.749902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:07.750280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:07.770865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:07.772185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:07.849911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1105/2245] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:40:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:07.904606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:07.906154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:08.015054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d5e2-4e9f-43a6-a8c2-9c2a7c6e02c3 service glance] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:08.022747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:08.023454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:08.023733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:08.024077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:08.034882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:08.036521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:08.046369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1141/2246] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:40:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:40:10.890769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:10.892197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:10.965433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4b79-74b6-4e8d-8909-9d4276bb54a0 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:10.973881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:10.974688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:10.974688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:10.974688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:10.980835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:11.041806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:11.050687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:11.098724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:11.102591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:11.129607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1106/2247] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:40:11.173821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd904-8368-4cb4-87ff-c7fdb6b315a8 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:11.181300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:11.182102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:11.182478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:11.182879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:11.185134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:11.292401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:11.294299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:11.304620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1142/2248] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:15.539947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:15.574581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4282f5-d866-4536-bd1c-815343430341 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:15.590272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:15.590989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:15.591414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:15.591743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:15.595978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:15.657297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:15.659328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17f47-f1ea-41b2-9830-6170f4ad8ff2 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:15.681512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1107/2249] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:15.909949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:15.912271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:15.941971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:15.983793 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055af0-7009-4476-957b-fe599f480f31 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:15.990032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:15.990733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:15.991024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:15.991297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:15.993723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:16.107831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:16.109293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:16.124162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1143/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:18.830555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ece4-1406-4983-8af4-5bef3db72dee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:18.832060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ece4-1406-4983-8af4-5bef3db72dee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:18.832599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ece4-1406-4983-8af4-5bef3db72dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:18.860066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-65a1ece4-1406-4983-8af4-5bef3db72dee None None] MFA Rules not processed for user `9dc07d7e2ff644c0942c2aaf5b498656`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:18.861572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-65a1ece4-1406-4983-8af4-5bef3db72dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:18.875269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1108/2251] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:19.533913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:19.555742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c87766-e395-4fb6-8f44-82fa88016a9d service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:19.568388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:19.570735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:19.571165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:19.571455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:19.577575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:19.600030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:19.601652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:19.619093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1144/2252] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:19.973688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:19.974121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:19.974243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:19.991519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] MFA Rules not processed for user `db8e18e9735c4cd0afb79921383e8b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:19.992647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:20.000241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1109/2253] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:20.198716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:20.200694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:20.293407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291314b3-d568-42bc-9fe6-d1802227b9af service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:20.298619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:20.299382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:20.299670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:20.299955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:20.301853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:20.315411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:20.317333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:20.330309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1145/2254] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:20.533774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:20.551936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e78e2-3abc-4f8f-97e8-f921afeb6c3a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:20.557418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:20.558759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:20.562870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:20.563220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:20.566740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:20.587624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:20.589827 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:20.601275 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1110/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:21.222722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:21.224863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:21.244358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a39ba-b570-448b-b370-7e17a6381325 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:21.250529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:21.251836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:21.252360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:21.252981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:21.255807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:21.275290 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:21.281236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:21.297410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1146/2256] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:21.340900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:21.342458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:21.362208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53baf8-dc09-457f-a29d-b6168527cc25 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:21.371198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:21.372134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:21.372620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:21.372888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] PATH_INFO: `/v3/users/b1430d9ff64e4a9fa2f40bb31a098090` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:21.374379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1430d9ff64e4a9fa2f40bb31a098090)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:21.376517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:21.429943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1430d9ff64e4a9fa2f40bb31a098090'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.430698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1430d9ff64e4a9fa2f40bb31a098090'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.431373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1430d9ff64e4a9fa2f40bb31a098090'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.439814 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1430d9ff64e4a9fa2f40bb31a098090'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.447469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dabbd503-44bd-425b-8826-8ef46335fe12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1430d9ff64e4a9fa2f40bb31a098090'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.466909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1111/2257] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:21 2022] DELETE /identity/v3/users/b1430d9ff64e4a9fa2f40bb31a098090 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:21.647957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:21.668421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df57f9d-79c9-40f6-8127-11464472a594 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:21.719508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:21.747842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:21.748728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:21.749007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:21.749275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] PATH_INFO: `/v3/projects/c41af6aa819743d380018ec1eed0a793` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:21.750545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41af6aa819743d380018ec1eed0a793)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:21.752493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:21.823267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41af6aa819743d380018ec1eed0a793', 'request_id': 'req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.829587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41af6aa819743d380018ec1eed0a793', 'request_id': 'req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.839044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41af6aa819743d380018ec1eed0a793', 'request_id': 'req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.863079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] The token cache is being invalidate because project c41af6aa819743d380018ec1eed0a793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:21.863373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.864045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f260eacf-ee3d-4c71-90b7-8ddacc1d02c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:21.871702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1147/2258] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:21 2022] DELETE /identity/v3/projects/c41af6aa819743d380018ec1eed0a793 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:21.876451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:21.877069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:21.877505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:21.904204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:21.905845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:21.988991 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1112/2259] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:21.997768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:21.998126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:21.998350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.020521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:22.021939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:22.030387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1148/2260] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:22.034666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.045791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50690618-eb5a-45ac-b9dc-0b19f01ca44c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.052755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.053300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.054511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.054511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.055074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:22.056155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.058226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.068516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:22.075159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1113/2261] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:22.076989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.087960 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7df5-04e6-4dc2-b215-dd1c5725fde5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.093449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.094169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.094514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.094800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.095542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.097794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.128740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1149/2262] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:22.133114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.143978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376675-3131-4617-9bdb-f16695ea30be admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.237333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.238280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.239394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.239812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.240661 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.244446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.286673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1114/2263] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:22.291065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.306195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a94e-0a1e-4804-adbc-048d40d6b4c9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.327494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.334626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.335140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.335591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.336466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.341516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e53ce-1a83-4781-b60b-0598fbc6f13f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.359914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1150/2264] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:22.365223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.380165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.380364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e66d3-0b54-4ac7-b013-f497b4ed3882 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.380878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.381682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.397287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.397287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.397845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.397845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.398397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.400676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.413848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:22.417292 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:22.431186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1115/2265] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:22.434623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.458552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8450-ae02-4c1b-bb94-5ee011d73cd9 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.471919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.473313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.473637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.473890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] PATH_INFO: `/v3/projects/496c2a737f634d42acfc80ec441221e8/users/e4b2c723cc4548dea6369e755fbc2a83/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.495504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1151/2266] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:40:22.504896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496c2a737f634d42acfc80ec441221e8, user_id=e4b2c723cc4548dea6369e755fbc2a83, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.508839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.510447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:22.511589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97f49c-ccd0-46e0-8e60-a1218cd8f414 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.529414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1116/2267] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:22 2022] PUT /identity/v3/projects/496c2a737f634d42acfc80ec441221e8/users/e4b2c723cc4548dea6369e755fbc2a83/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:22.541288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.541738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.542242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.558425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:22.577973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] MFA Rules not processed for user `e4b2c723cc4548dea6369e755fbc2a83`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:22.579853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:22.613575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123107-a431-42e7-98b6-cc44be712c6a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.667009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1117/2268] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:22.671648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.685371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.686162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.686447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.686889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.747220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:22.793181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f64b1-dcf9-4c0b-964c-d1099c095025 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:22.793962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.797939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.799188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:22.799967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:22.800349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:22.800724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:22.801552 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:22.803735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:22.810679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1152/2269] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:40:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 00:40:22.853859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1118/2270] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/projects => generated 389 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:40:22.857789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:22.877753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223581f-8ccd-43bf-876e-b07501cc43e0 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.040623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.041485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.041766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.042072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.042805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.068775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.128221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1153/2271] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:22 2022] POST /identity/v3/users => generated 301 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:40:23.132304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:23.150020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3e91f-cd48-4349-a4ad-6206c9feb806 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.177095 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.177918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.178308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.178686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.180892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.183089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.215694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1119/2272] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:40:23.219974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:23.228816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:23.234833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a6ea-5f4b-4d75-b08e-5a8f9aca33a0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.241418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.251924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.252542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.252987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] PATH_INFO: `/v3/projects/2f3989fd56424e5c89c00bf653f6a280/users/f6639e56da47461fa2c048437e29f394/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.288495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3989fd56424e5c89c00bf653f6a280, user_id=f6639e56da47461fa2c048437e29f394, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.300789 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.341877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1154/2273] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:23 2022] PUT /identity/v3/projects/2f3989fd56424e5c89c00bf653f6a280/users/f6639e56da47461fa2c048437e29f394/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:23.348464 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:23.369477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1031e9-65fd-4207-8f6a-ca4de545a8d8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.547093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.547789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.548071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.548348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.552902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.554608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6a898-1abf-477b-b3d8-ca04cfa64fc0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.578922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1120/2274] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:23.583074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:23.600488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf81c5-0abb-4e68-8c41-fe869110e07a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.609474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.611840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.612123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.612413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] PATH_INFO: `/v3/projects/2f3989fd56424e5c89c00bf653f6a280/users/f6639e56da47461fa2c048437e29f394/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.614676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3989fd56424e5c89c00bf653f6a280, user_id=f6639e56da47461fa2c048437e29f394, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.617413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.629639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1155/2275] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:23 2022] PUT /identity/v3/projects/2f3989fd56424e5c89c00bf653f6a280/users/f6639e56da47461fa2c048437e29f394/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:23.643608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.643894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.644174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.679502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] MFA Rules not processed for user `f6639e56da47461fa2c048437e29f394`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:23.681588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:23.802830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1121/2276] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:23.810063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:23.812083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:23.916106 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1854-fdef-48d7-a7af-3099029082c6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:23.925379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:23.926020 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:23.926420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:23.926706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:23.949626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:23.954632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297fe9a-56ba-43d5-862a-5e3c38f1e797 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:23.974272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1156/2277] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:24.214628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:24.277816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:24.340779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a154c1d-e1e5-428b-bad1-3a265490df22 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:24.347718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:24.348740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:24.349301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:24.349676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] PATH_INFO: `/v3/users/e4b2c723cc4548dea6369e755fbc2a83` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:24.351849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b2c723cc4548dea6369e755fbc2a83)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:24.355754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:24.459729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b2c723cc4548dea6369e755fbc2a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:24.463207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b2c723cc4548dea6369e755fbc2a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:24.464178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b2c723cc4548dea6369e755fbc2a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:24.486718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b2c723cc4548dea6369e755fbc2a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:24.505501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b2c723cc4548dea6369e755fbc2a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:24.567166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1122/2278] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:24 2022] DELETE /identity/v3/users/e4b2c723cc4548dea6369e755fbc2a83 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:24.571129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:24.571585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:24.571692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:24.600081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:24.601192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:24.796303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1157/2279] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:24.825336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:24.827109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:24.987782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4dd3a-25fe-412c-a93f-f7f123307275 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:25.001204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:25.001960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:25.002336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:25.004782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:25.037680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:25.039364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:25.069566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1123/2280] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:25.746795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:25.748346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:25.777489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934db58a-28a7-4991-adb8-5f445d2b6488 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:25.807566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:25.809329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:25.809667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:25.810003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] PATH_INFO: `/v3/projects/496c2a737f634d42acfc80ec441221e8` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:25.811970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=496c2a737f634d42acfc80ec441221e8)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:25.814337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:25.940695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496c2a737f634d42acfc80ec441221e8', 'request_id': 'req-5919b249-7326-43f7-8874-b2f4f0413c0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:25.952325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496c2a737f634d42acfc80ec441221e8', 'request_id': 'req-5919b249-7326-43f7-8874-b2f4f0413c0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:25.968865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496c2a737f634d42acfc80ec441221e8', 'request_id': 'req-5919b249-7326-43f7-8874-b2f4f0413c0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:25.970536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:25.992419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] The token cache is being invalidate because project 496c2a737f634d42acfc80ec441221e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:25.992979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:25.994200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:25.996320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1158/2281] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:25 2022] DELETE /identity/v3/projects/496c2a737f634d42acfc80ec441221e8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:26.000121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:26.017937 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd94ab-5c65-4f1e-8171-caeb616ac211 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:26.216336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:26.217294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:26.217616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:26.218553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] PATH_INFO: `/v3/users/f6639e56da47461fa2c048437e29f394` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:26.221344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6639e56da47461fa2c048437e29f394)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:26.226441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:26.235649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:26.316792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6639e56da47461fa2c048437e29f394'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.317568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6639e56da47461fa2c048437e29f394'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.318420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6639e56da47461fa2c048437e29f394'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.335689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6639e56da47461fa2c048437e29f394'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.361732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-1f8f8894-6eb6-45ca-a61f-71c1265d8d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6639e56da47461fa2c048437e29f394'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.385026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1124/2282] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:25 2022] DELETE /identity/v3/users/f6639e56da47461fa2c048437e29f394 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:26.824342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:26.825812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:26.840084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919b249-7326-43f7-8874-b2f4f0413c0e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:26.909017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:26.909795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:26.910078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:26.910353 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] PATH_INFO: `/v3/projects/2f3989fd56424e5c89c00bf653f6a280` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:26.911586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3989fd56424e5c89c00bf653f6a280)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:26.913498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:26.971405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3989fd56424e5c89c00bf653f6a280', 'request_id': 'req-8405378b-79da-453a-92f7-8350f097f0af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.977852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3989fd56424e5c89c00bf653f6a280', 'request_id': 'req-8405378b-79da-453a-92f7-8350f097f0af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:26.986577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3989fd56424e5c89c00bf653f6a280', 'request_id': 'req-8405378b-79da-453a-92f7-8350f097f0af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:27.000154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] The token cache is being invalidate because project 2f3989fd56424e5c89c00bf653f6a280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:27.000439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:27.001117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8405378b-79da-453a-92f7-8350f097f0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:27.002683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1159/2283] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:26 2022] DELETE /identity/v3/projects/2f3989fd56424e5c89c00bf653f6a280 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:27.007628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.008156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.008377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.030739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:27.032067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:27.094591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1125/2284] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:27.103155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.103445 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.103722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.121954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:27.123191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:27.131951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1160/2285] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:27.136786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.147516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834662b5-7bf6-4dd1-9d92-e5a3097f8025 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.153026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.153698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.153998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.154278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.155040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:27.155349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.156962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.170889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1126/2286] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:27.175017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.184620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809c452-a93f-480e-a48d-e087aa058863 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.189371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.190023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.190303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.190579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.191319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.193234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.222075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1161/2287] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:27.228439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.239151 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e8a8a-eab3-46fa-837b-c7f862457d23 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.296515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.298363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:27.303120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.303919 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.304346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.304624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.305470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.307458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.324734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:27.337424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1127/2288] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:27.341517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.359292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dec702-f4f3-4628-9f4a-615c4b9fe29b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.364882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.365547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.365834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.366106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.366865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.368500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.416869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1128/2289] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:40:27.417585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:27.419870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.434802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765f19-8bf2-48f1-a52c-19656a120b67 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.440049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.440886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.441176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.441453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.441625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d063a1e-58d2-4aea-850d-f7abc4cfd411 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.442191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.448395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.449165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.449654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.450019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.452115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:27.474612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.496030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1129/2290] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:27.497548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.518231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98d94f-6363-4c94-8092-dc55885a9ca1 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.532868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.534086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.536182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.536967 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] PATH_INFO: `/v3/projects/cbac6ef83fe04212a83a8ec2e41fc95d/users/814f37f10f8e4868b9072e155bde5a83/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.575392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbac6ef83fe04212a83a8ec2e41fc95d, user_id=814f37f10f8e4868b9072e155bde5a83, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.578748 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef82e0-6a20-4931-9fbb-ba825b812b69 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.593079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.597086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1130/2291] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:27 2022] PUT /identity/v3/projects/cbac6ef83fe04212a83a8ec2e41fc95d/users/814f37f10f8e4868b9072e155bde5a83/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:27.597756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.608406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1162/2292] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:40:27.608632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.608867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.609251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.652395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] MFA Rules not processed for user `814f37f10f8e4868b9072e155bde5a83`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:27.653832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:27.769015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1131/2293] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:27.774147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:27.795649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82db60-aed8-4ad4-adc6-4bea0830b2dc service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:27.917630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:27.919035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:27.919538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:27.920084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:27.920988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:27.923833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:27.985705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1163/2294] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/projects => generated 367 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:27.989895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.012529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31375-b71a-455c-abb2-b5b0861b1009 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.108697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.109482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.109864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.110362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.111216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.127498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.154881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1132/2295] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:27 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:28.159164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.170140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80001590-ca84-4d58-b874-7913d4c986b9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.175328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.176101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.176377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.176670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.177409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.179044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.192929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1164/2296] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:28.197054 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.207715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b316-405e-4d74-b890-f3f2e50cf895 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.212474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.213348 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.213632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.213906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] PATH_INFO: `/v3/projects/1d4afc41ec0047bab4678c02ca4412f7/users/8cfbf9d86f2e4e749ea8b282957e7a55/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.229527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4afc41ec0047bab4678c02ca4412f7, user_id=8cfbf9d86f2e4e749ea8b282957e7a55, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.234463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.251014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1133/2297] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:28 2022] PUT /identity/v3/projects/1d4afc41ec0047bab4678c02ca4412f7/users/8cfbf9d86f2e4e749ea8b282957e7a55/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:28.252277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.263228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c39aae-f776-4954-aab3-337ecf67ad6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.320086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.320783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.321066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.321346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.322070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.323706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88c9e-33d6-405e-9719-a4cbcbbbc004 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.340056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1165/2298] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:28.343974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.355858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c4932-3c72-41db-8bad-924bc36848d0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.360853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.361740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.362020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.362297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] PATH_INFO: `/v3/projects/1d4afc41ec0047bab4678c02ca4412f7/users/8cfbf9d86f2e4e749ea8b282957e7a55/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.364527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4afc41ec0047bab4678c02ca4412f7, user_id=8cfbf9d86f2e4e749ea8b282957e7a55, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.367317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.370345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:28.383852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1134/2299] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:28 2022] PUT /identity/v3/projects/1d4afc41ec0047bab4678c02ca4412f7/users/8cfbf9d86f2e4e749ea8b282957e7a55/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:28.395067 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.395389 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.395672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.402095 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:28.431998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] MFA Rules not processed for user `8cfbf9d86f2e4e749ea8b282957e7a55`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:28.433778 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:28.517219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1166/2300] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:28.526860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.528915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:28.593472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3081d19-6bab-4308-a69c-d00eb2383eb8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.597200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.598093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.598372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.598663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.608520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.609977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.619397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1135/2301] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:28.700195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:28.710212 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ac8b-f919-45f7-8b76-c8de81971db0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:28.714705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:28.715470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:28.715877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:28.716161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:28.798304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:28.800124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bb15f-7fc1-466d-b456-fbbd2e9a1407 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:28.811824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1167/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:33.894308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:33.957015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a18e61-b48f-436b-9643-0f4f3af02e37 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:33.961711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:33.962571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:33.962879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:33.963148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] PATH_INFO: `/v3/users/814f37f10f8e4868b9072e155bde5a83` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:33.964333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=814f37f10f8e4868b9072e155bde5a83)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:33.966267 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:34.012739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814f37f10f8e4868b9072e155bde5a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:34.013380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814f37f10f8e4868b9072e155bde5a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:34.013978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814f37f10f8e4868b9072e155bde5a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:34.021469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814f37f10f8e4868b9072e155bde5a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:34.029422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814f37f10f8e4868b9072e155bde5a83'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:34.047637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1136/2303] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:33 2022] DELETE /identity/v3/users/814f37f10f8e4868b9072e155bde5a83 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:34.055507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:34.056035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:34.056137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:34.072678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:34.073871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:34.132090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1168/2304] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:34.141561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:34.142986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:34.161675 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:34.213200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709c162-dcdc-414b-b5c6-9dac27d249ed admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:34.216955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:34.218444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:34.218942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:34.219230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:34.231359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:34.232809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:34.243760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1137/2305] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:35.183077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:35.184708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:35.197423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280e51-b21b-4aeb-beab-d8ba6971447a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:35.202301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:35.203267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:35.203639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:35.203917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] PATH_INFO: `/v3/projects/cbac6ef83fe04212a83a8ec2e41fc95d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:35.205127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbac6ef83fe04212a83a8ec2e41fc95d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:35.207081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:35.235348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:35.280555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbac6ef83fe04212a83a8ec2e41fc95d', 'request_id': 'req-93363c78-d7bd-4eeb-b826-03b45b00a6dd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.286381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbac6ef83fe04212a83a8ec2e41fc95d', 'request_id': 'req-93363c78-d7bd-4eeb-b826-03b45b00a6dd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.295601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbac6ef83fe04212a83a8ec2e41fc95d', 'request_id': 'req-93363c78-d7bd-4eeb-b826-03b45b00a6dd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.310109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] The token cache is being invalidate because project cbac6ef83fe04212a83a8ec2e41fc95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:35.310409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.311146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.313143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1169/2306] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:35 2022] DELETE /identity/v3/projects/cbac6ef83fe04212a83a8ec2e41fc95d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:35.317137 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:35.328749 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecc6e5-bb39-4fd3-a60e-e98768d6c3c6 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:35.396615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:35.397844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:35.398485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:35.398992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] PATH_INFO: `/v3/users/8cfbf9d86f2e4e749ea8b282957e7a55` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:35.400493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cfbf9d86f2e4e749ea8b282957e7a55)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:35.403167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:35.472989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cfbf9d86f2e4e749ea8b282957e7a55'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.479188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cfbf9d86f2e4e749ea8b282957e7a55'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.480062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cfbf9d86f2e4e749ea8b282957e7a55'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.493543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cfbf9d86f2e4e749ea8b282957e7a55'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.508510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f5188ec-7d49-42eb-b789-383e5fce7ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cfbf9d86f2e4e749ea8b282957e7a55'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:35.535243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1138/2307] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:35 2022] DELETE /identity/v3/users/8cfbf9d86f2e4e749ea8b282957e7a55 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:35.900765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:35.902450 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:35.927346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93363c78-d7bd-4eeb-b826-03b45b00a6dd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:35.991706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:35.992754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:35.993030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:35.993474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] PATH_INFO: `/v3/projects/1d4afc41ec0047bab4678c02ca4412f7` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:35.994767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4afc41ec0047bab4678c02ca4412f7)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:35.996704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.060113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4afc41ec0047bab4678c02ca4412f7', 'request_id': 'req-64ba044d-9c9e-4880-b95f-edd33735bf6e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:36.069064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4afc41ec0047bab4678c02ca4412f7', 'request_id': 'req-64ba044d-9c9e-4880-b95f-edd33735bf6e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:36.080435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4afc41ec0047bab4678c02ca4412f7', 'request_id': 'req-64ba044d-9c9e-4880-b95f-edd33735bf6e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:36.095592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] The token cache is being invalidate because project 1d4afc41ec0047bab4678c02ca4412f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:36.095884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:36.096570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64ba044d-9c9e-4880-b95f-edd33735bf6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:36.098879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1170/2308] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:35 2022] DELETE /identity/v3/projects/1d4afc41ec0047bab4678c02ca4412f7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:36.104474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.104806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.105085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.133660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:36.135173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:36.184024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:36.219617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1139/2309] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:36.228567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.229101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.229101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.271516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:36.272762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:36.297725 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1171/2310] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:36.302109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.322496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb67ec-3d57-409d-b039-fb2b3f9b7a62 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.333042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.334354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.334641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.334980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.335735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:36.336643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.338331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.357905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1140/2311] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:36.362325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.372557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4483d-0d56-4c41-b3b6-b0db0b0df80f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.378590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.379756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.379756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.379992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.380488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.382428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.412214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1172/2312] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:36.413344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.423712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a963e8c-d040-447b-8abb-2d86c3f5076d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.493405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.493405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.493405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.493405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.493856 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.493943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.518973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1141/2313] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:40:36.523060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.534957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28db247-f8ba-46ce-ab88-390f18d6e443 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.540489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.541491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.541772 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.542089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.542859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.544565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.552614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:36.561177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1173/2314] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:40:36.565472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.576780 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e316c3b-de8e-4a3a-969e-df6f6114f370 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.581722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.582446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.582663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.582977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.583692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.585287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bb0de-1fea-4572-b06e-0204edeb23c3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.597410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1142/2315] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:36.601193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.611520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6a27e-6153-4c11-af98-bb196bf2a7a8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.616382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.617882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.618250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.618580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] PATH_INFO: `/v3/projects/ca72d0d923724d43bf059858c281b00b/users/3e26c4d35f594cd5ab6ec3069c2ed377/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.640185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca72d0d923724d43bf059858c281b00b, user_id=3e26c4d35f594cd5ab6ec3069c2ed377, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.642938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.656282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1174/2316] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:36 2022] PUT /identity/v3/projects/ca72d0d923724d43bf059858c281b00b/users/3e26c4d35f594cd5ab6ec3069c2ed377/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:36.667590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.667963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.668245 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.696629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] MFA Rules not processed for user `3e26c4d35f594cd5ab6ec3069c2ed377`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:36.697878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:36.771657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1143/2317] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:36.775879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.846640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61500ee3-64bb-4bce-9f5c-77a55236a6fd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.851864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.852596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.852947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.853287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.853996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.856032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.883829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1175/2318] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:36.887867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:36.899087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4576d-ef32-46f7-b968-5c6f1d860877 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:36.928387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:36.959196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:36.959867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:36.960146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:36.960421 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:36.961326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:36.964056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:36.989816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1144/2319] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:36 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:36.990712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:37.002167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4787-a6d2-4eeb-85cf-22090d3ba1d1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:37.007431 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:37.008207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.008491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.008768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.009528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:37.011287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:37.022732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1176/2320] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:37.026987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:37.037585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e618a3-19b7-4a6c-9ca9-74fb8fe25e4b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:37.042872 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:37.043754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.044035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.044308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] PATH_INFO: `/v3/projects/05b41081c3014c62b35058d55018e55a/users/b6263fd3030a4e9d8f55bc5897293620/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.060677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b41081c3014c62b35058d55018e55a, user_id=b6263fd3030a4e9d8f55bc5897293620, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:37.063405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:37.078536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1145/2321] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:37 2022] PUT /identity/v3/projects/05b41081c3014c62b35058d55018e55a/users/b6263fd3030a4e9d8f55bc5897293620/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:37.082437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:37.092589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6fd56-3e74-426c-b792-98560911517f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:37.161311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:37.162333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.162852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.163326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.164886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:37.166938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505034ea-ac4c-4eed-8dc2-4c208a6d014b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:37.179446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1177/2322] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:37.183979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:37.195353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156e03-462f-48e6-8371-5d68ba669475 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:37.200526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:37.201793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.202336 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.202886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] PATH_INFO: `/v3/projects/05b41081c3014c62b35058d55018e55a/users/b6263fd3030a4e9d8f55bc5897293620/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.205197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b41081c3014c62b35058d55018e55a, user_id=b6263fd3030a4e9d8f55bc5897293620, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:37.208170 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:37.219970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1146/2323] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:37 2022] PUT /identity/v3/projects/05b41081c3014c62b35058d55018e55a/users/b6263fd3030a4e9d8f55bc5897293620/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:37.231104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.231673 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.232024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.260874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] MFA Rules not processed for user `b6263fd3030a4e9d8f55bc5897293620`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:37.262091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:37.296499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:37.333883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1178/2324] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:37.345065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:37.346800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:37.408262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bec5a-ede2-475b-9d79-d0c5502bf969 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:37.412266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:37.412908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:37.413187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:37.413470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:37.423764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:37.425215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c092-9022-4328-a7f4-9a7414379d76 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:37.436225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1147/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:38.327956 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:38.391784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dee771-0640-423b-a3cc-e505c59fc2fa None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:38.398124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:38.399755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:38.399755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:38.399755 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] PATH_INFO: `/v3/users/3e26c4d35f594cd5ab6ec3069c2ed377` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:38.400390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e26c4d35f594cd5ab6ec3069c2ed377)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:38.402312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:38.448758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e26c4d35f594cd5ab6ec3069c2ed377'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:38.449478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e26c4d35f594cd5ab6ec3069c2ed377'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:38.455946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e26c4d35f594cd5ab6ec3069c2ed377'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:38.469273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e26c4d35f594cd5ab6ec3069c2ed377'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:38.500197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e26c4d35f594cd5ab6ec3069c2ed377'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:38.507594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1179/2326] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:38 2022] DELETE /identity/v3/users/3e26c4d35f594cd5ab6ec3069c2ed377 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:38.511866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:38.512325 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:38.512446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:38.530303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:38.531549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:38.608148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1148/2327] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:38.618202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:38.619814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:38.694664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c0459-4ff9-4cf2-ab4d-c19075258e22 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:38.698638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:38.699392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:38.699744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:38.700086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:38.707396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:38.713157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:38.715646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:38.733727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1180/2328] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:39.163053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:39.164532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:39.176400 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af810a6-d8e9-4eb8-9499-f373e8d5bfee None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:39.181736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:39.182660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:39.182904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:39.183322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] PATH_INFO: `/v3/projects/ca72d0d923724d43bf059858c281b00b` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:39.184745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca72d0d923724d43bf059858c281b00b)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:39.186657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:39.215185 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:39.241078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca72d0d923724d43bf059858c281b00b', 'request_id': 'req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.247183 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca72d0d923724d43bf059858c281b00b', 'request_id': 'req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.262427 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca72d0d923724d43bf059858c281b00b', 'request_id': 'req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.274575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] The token cache is being invalidate because project ca72d0d923724d43bf059858c281b00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:39.274895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.275583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.277368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1149/2329] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:39 2022] DELETE /identity/v3/projects/ca72d0d923724d43bf059858c281b00b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:39.281167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:39.291276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971730a2-c6ad-4ab6-9840-1817fd3b8db5 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:39.347911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:39.348787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:39.349068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:39.349344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] PATH_INFO: `/v3/users/b6263fd3030a4e9d8f55bc5897293620` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:39.350551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6263fd3030a4e9d8f55bc5897293620)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:39.353009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:39.402305 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6263fd3030a4e9d8f55bc5897293620'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.402989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6263fd3030a4e9d8f55bc5897293620'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.410421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6263fd3030a4e9d8f55bc5897293620'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.419014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6263fd3030a4e9d8f55bc5897293620'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.433843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6263fd3030a4e9d8f55bc5897293620'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:39.440476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1181/2330] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:39 2022] DELETE /identity/v3/users/b6263fd3030a4e9d8f55bc5897293620 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:39.801379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:39.803230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:39.897996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77fa6d-a6f4-4415-a7d1-9c5c06053c6c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:39.948990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:39.950677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.021126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.022253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.022947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.024441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.026660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.041476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.043086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd414b4-b519-4e64-af2e-ba09e33c9400 service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.053874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1150/2331] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:40.073206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac9e5-e75c-40d9-944e-f0828cb99d72 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.078182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.079167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.079534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.079876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] PATH_INFO: `/v3/projects/05b41081c3014c62b35058d55018e55a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.081166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b41081c3014c62b35058d55018e55a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.083276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.105545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:40.179620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b41081c3014c62b35058d55018e55a', 'request_id': 'req-0d203ee1-abd7-4252-ab16-df3652283af2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:40.199468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b41081c3014c62b35058d55018e55a', 'request_id': 'req-0d203ee1-abd7-4252-ab16-df3652283af2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:40.217420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b41081c3014c62b35058d55018e55a', 'request_id': 'req-0d203ee1-abd7-4252-ab16-df3652283af2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:40.240381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] The token cache is being invalidate because project 05b41081c3014c62b35058d55018e55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:40.240844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:40.241843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0d203ee1-abd7-4252-ab16-df3652283af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:40.249705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.250868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.250868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.253113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1182/2332] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:39 2022] DELETE /identity/v3/projects/05b41081c3014c62b35058d55018e55a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:40:40.273915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:40.276111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.322072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:40.362750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1151/2333] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:40.364792 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.365803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.366438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.387903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:40.389116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.398313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1183/2334] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:40.402585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.412604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0df52-d22c-4aea-a6bc-c1b5c2abfe7f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.417891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.418562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.418908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.419186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.420086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:40.420399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.422084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.439149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1152/2335] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:40.440410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.450610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f136a-9361-486b-9f50-b31b82c361fb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.455289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.455936 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.456217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.456493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.457181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.459145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.489899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1184/2336] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:40.495520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.505904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114991ff-eefd-4caf-8a02-2e72f9a74fda admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.575599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.576384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.576731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.577072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.577829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.579875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.604157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1153/2337] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:40.605412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.617565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1abd68-535d-487f-9449-2ea225144311 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.623054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.623800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.624156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.624606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.625419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.627242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.639502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1185/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:40.643939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.654673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33e4b-ae91-461c-b85c-ac9864da39d5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.660035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.660505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.660907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.661045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.661941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.663627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babf75-6031-41cf-bd72-d1451ec316c7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.674960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1154/2339] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:40.679169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.690092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47673e7c-fb44-458c-8554-249b5ea8a80e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.694733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.696311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.696686 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.696959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] PATH_INFO: `/v3/projects/8b2f53250e6746a78ab164530c1ddd60/users/bcef68a54a7d49749bf612b3153f72fe/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.713592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2f53250e6746a78ab164530c1ddd60, user_id=bcef68a54a7d49749bf612b3153f72fe, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.716337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4ce0-8afa-41cf-823d-665f061596fb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.731735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1186/2340] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:40 2022] PUT /identity/v3/projects/8b2f53250e6746a78ab164530c1ddd60/users/bcef68a54a7d49749bf612b3153f72fe/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:40.740820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.741110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.741387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.744365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.744657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.744934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.759981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] MFA Rules not processed for user `eceb9f042cb24343892f09454c7e685a`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:40.761178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.771745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] MFA Rules not processed for user `bcef68a54a7d49749bf612b3153f72fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:40.772909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.816666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1155/2341] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:40.839158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1187/2342] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:40.850254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.852158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:40.904401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:40.910830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370695c1-7e06-48c1-88dd-26f040af22fa None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:40.917242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:40.917890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:40.918168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:40.918440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:40.927391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:40.928824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe950b0c-d286-4a16-acb8-eb090590fe0f service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:40.938629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1156/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:40.958138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:40.990601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:41.040203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc19ce2-d611-42c2-ab95-2cffa75e0005 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:41.044911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:41.045786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:41.046064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:41.046333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] PATH_INFO: `/v3/users/bcef68a54a7d49749bf612b3153f72fe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:41.047560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcef68a54a7d49749bf612b3153f72fe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:41.049709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:41.104799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcef68a54a7d49749bf612b3153f72fe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.105591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcef68a54a7d49749bf612b3153f72fe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.114384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcef68a54a7d49749bf612b3153f72fe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.123501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcef68a54a7d49749bf612b3153f72fe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.140602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcef68a54a7d49749bf612b3153f72fe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.151790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1188/2344] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:40 2022] DELETE /identity/v3/users/bcef68a54a7d49749bf612b3153f72fe => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:41.154187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:41.154566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:41.154964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:41.176947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:41.178162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:41.237804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1157/2345] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:41.247647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:41.249160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:41.307978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defa968-57f5-4d25-9738-d175ac00f8f3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:41.311683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:41.312329 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:41.312614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:41.312889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:41.322536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:41.324055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed9477-ac3e-4b0e-95fb-a5a68652e61a service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:41.334093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1189/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:41.824149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:41.826232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:41.841297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2345c-c1d9-444a-b64c-a349fdacf8e2 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:41.854921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:41.855840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:41.856115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:41.856414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] PATH_INFO: `/v3/projects/8b2f53250e6746a78ab164530c1ddd60` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:41.857769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2f53250e6746a78ab164530c1ddd60)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:41.859785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:41.908993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2f53250e6746a78ab164530c1ddd60', 'request_id': 'req-e0399eaa-248a-4def-be3a-746e3089d6eb'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.914868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2f53250e6746a78ab164530c1ddd60', 'request_id': 'req-e0399eaa-248a-4def-be3a-746e3089d6eb'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.922716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2f53250e6746a78ab164530c1ddd60', 'request_id': 'req-e0399eaa-248a-4def-be3a-746e3089d6eb'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.935505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] The token cache is being invalidate because project 8b2f53250e6746a78ab164530c1ddd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:41.935798 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.936448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e0399eaa-248a-4def-be3a-746e3089d6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:41.938191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1158/2347] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:41 2022] DELETE /identity/v3/projects/8b2f53250e6746a78ab164530c1ddd60 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:41.943030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:41.943523 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:41.943614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:41.968674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:41.969898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:42.017054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:42.035468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1190/2348] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:42.042915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.043196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.043472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.063671 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:42.064906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:42.073585 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1159/2349] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:42.077981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.088060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830b013-f597-4552-89dc-2dbed0cca62e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.092752 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.093744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.093744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.094088 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.094703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:42.095043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.096651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.113259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1191/2350] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:42.118129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.128167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2d7e3-703f-471c-856c-dcf815094af9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.133612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.134263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.134539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.134867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.135574 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.138020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.166641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1160/2351] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:40:42.167749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.178022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c139b-4878-41c3-ab58-f3855b6c7352 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.255829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.256566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.256853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.257130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.257876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.259938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.284233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1192/2352] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:42 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:42.289809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.302847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628889-c331-4bf1-9991-18695eb9712f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.309665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.310342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.310623 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.310938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.311702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.313341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.326655 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1161/2353] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:42.330522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.342368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e7a1-cc2f-401c-993f-a84d49d79ae9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.348235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.349167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.349487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.349914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.350687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.353265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80bc4f-5287-4258-82ce-985321e2eff7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.365790 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1193/2354] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:42.370637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.380670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0edbd-04c6-44c4-a893-e305fe65eda6 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.385255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.386115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.386391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.386660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] PATH_INFO: `/v3/projects/cb1ed90f59324bd995878ce5d76e6787/users/1f7dacb8a165437c9524a38cc807db9a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.400892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1ed90f59324bd995878ce5d76e6787, user_id=1f7dacb8a165437c9524a38cc807db9a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.403571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.416184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1162/2355] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:42 2022] PUT /identity/v3/projects/cb1ed90f59324bd995878ce5d76e6787/users/1f7dacb8a165437c9524a38cc807db9a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:42.426288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.426582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.426911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.453506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] MFA Rules not processed for user `1f7dacb8a165437c9524a38cc807db9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:42.454672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:42.519318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1194/2356] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:42.530053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:42.532208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:42.544662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:42.617847 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c61b-77a5-4536-843d-07dd84f075c5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:42.624618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:42.625352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:42.625639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:42.625945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:42.635236 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:42.636699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1673b3-af7d-4bbe-a256-c0a4fe0b7687 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:42.659717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1163/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:51.220097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:51.269864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:51.286101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952cc17-8d7f-4702-9f17-0ab843fcd2e5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:51.291167 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:51.292039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:51.292315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:51.292587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] PATH_INFO: `/v3/users/1f7dacb8a165437c9524a38cc807db9a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:51.293785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7dacb8a165437c9524a38cc807db9a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:51.295751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:51.344316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7dacb8a165437c9524a38cc807db9a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.344994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7dacb8a165437c9524a38cc807db9a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.354855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7dacb8a165437c9524a38cc807db9a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.362991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7dacb8a165437c9524a38cc807db9a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.377195 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7dacb8a165437c9524a38cc807db9a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.383762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1195/2358] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:40:51 2022] DELETE /identity/v3/users/1f7dacb8a165437c9524a38cc807db9a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:51.388688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:51.389498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:51.389836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:51.410375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:51.411764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:51.473575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1164/2359] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:51.483471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:51.484878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:51.545383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf76f1-0102-4cc3-b9f0-1531a8fb3cd2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:51.549150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:51.549884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:51.550160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:51.550432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:51.560284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:51.561710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f47bd-e357-444d-ac62-88be9f8fe705 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:51.574281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1196/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:40:51.861815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:51.863510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:51.882201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0b91-d853-4698-938b-8a20f0800796 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:51.887297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:51.888342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:51.888617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:51.888957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] PATH_INFO: `/v3/projects/cb1ed90f59324bd995878ce5d76e6787` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:51.890246 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1ed90f59324bd995878ce5d76e6787)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:51.892306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:51.945974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1ed90f59324bd995878ce5d76e6787', 'request_id': 'req-f90b38d2-d576-4b8e-8334-e22c11d1898b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.952124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1ed90f59324bd995878ce5d76e6787', 'request_id': 'req-f90b38d2-d576-4b8e-8334-e22c11d1898b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.959195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1ed90f59324bd995878ce5d76e6787', 'request_id': 'req-f90b38d2-d576-4b8e-8334-e22c11d1898b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.973412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] The token cache is being invalidate because project cb1ed90f59324bd995878ce5d76e6787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:40:51.973702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.974351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f90b38d2-d576-4b8e-8334-e22c11d1898b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:40:51.976218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1165/2361] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:40:51 2022] DELETE /identity/v3/projects/cb1ed90f59324bd995878ce5d76e6787 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:51.981015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:51.981379 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:51.981717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:51.999560 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:52.000961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.068531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1197/2362] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:52.078024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.078387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.078731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.098287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:52.099538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.107605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1166/2363] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:52.111988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.121609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04ebed-eb9d-43df-9c35-5bdc996b1db9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.126241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.126945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.127229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.127503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.128242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:40:52.128551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.130155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.132617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:52.147315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1198/2364] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:40:52.153176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.162481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e478-efaf-4eaf-8cca-b71977e2bb04 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.168322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.169132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.169269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.169548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.170203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.172172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.200559 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1167/2365] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:40:52 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:52.204308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.214196 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf42a48-1c1d-4bf7-86cc-e1abe71e7058 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.271172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.271847 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.272129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.272403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.273099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.275593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.300539 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1199/2366] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:40:52 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:40:52.302034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.312822 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9165c8-e709-4b67-bb1d-2a4caebb8498 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.318106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.318763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.319074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.319347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.320076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.322191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.323118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:52.333812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1168/2367] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:52.338718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.349465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3bf07-0448-4d6e-a641-24cc30d21604 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.355470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.356135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.356420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.356811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.357873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.359230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed04fa-9b7b-4da1-98ae-24cfe3dd6d6c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.372262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1200/2368] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:40:52.375425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.385330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687d53-6e3e-41e2-a280-31b4ef7f9e19 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.390232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.391197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.391476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.392360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] PATH_INFO: `/v3/projects/e44c61e61323479fbabcbc28a5b29995/users/cf48ed606dfa4e20adca3372c10b97cb/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.407988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44c61e61323479fbabcbc28a5b29995, user_id=cf48ed606dfa4e20adca3372c10b97cb, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.410924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.433993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1169/2369] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:40:52 2022] PUT /identity/v3/projects/e44c61e61323479fbabcbc28a5b29995/users/cf48ed606dfa4e20adca3372c10b97cb/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:40:52.435769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff4b14-3d9e-45ff-8e0c-8403bd700fca None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.436062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff4b14-3d9e-45ff-8e0c-8403bd700fca None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.436342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff4b14-3d9e-45ff-8e0c-8403bd700fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.464239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-98ff4b14-3d9e-45ff-8e0c-8403bd700fca None None] MFA Rules not processed for user `cf48ed606dfa4e20adca3372c10b97cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:52.465380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-98ff4b14-3d9e-45ff-8e0c-8403bd700fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.536414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1201/2370] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:40:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:40:52.547468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.549143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.606864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a794-896d-4267-9993-dd6daf7e7ddb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.610698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.611410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.611691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.611968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.621290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.622723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.632040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1170/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:40:52.722377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.722675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.722999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.741008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:40:52.742221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.803429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1202/2372] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:40:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:40:52.816260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:52.817792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:52.895612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ba4f7-e251-4ff6-bddf-4707a524d899 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:52.901725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:52.902521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:52.902957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:52.903332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:52.912262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:52.914116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922a477-56b6-4f93-b7f3-3731d2eaa589 service swift] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:52.925215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1171/2373] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:40:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:40:57.816614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:40:57.820040 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:57.850655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:40:57.916796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c8439-3893-4dc8-8f02-963bb50b45dd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:40:57.923297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:40:57.923970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:40:57.924399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:40:57.924682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:40:57.926655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:40:58.069148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:40:58.070657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9770d4-7680-48f0-af88-1294c7927525 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:40:58.082994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1203/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:40:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:02.920975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:08.077584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=46 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:09.506408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.506973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.507071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.508529 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1172/2375] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:41:09.514728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73560-30a2-4d12-bcc7-b9d17339ae8d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.515225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73560-30a2-4d12-bcc7-b9d17339ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.515343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73560-30a2-4d12-bcc7-b9d17339ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.536619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-81b73560-30a2-4d12-bcc7-b9d17339ae8d None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:09.537860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-81b73560-30a2-4d12-bcc7-b9d17339ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:09.617617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1204/2376] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:41:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:41:09.629678 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:09.631408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:09.695040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d82f-a5eb-49e2-910e-05c9b57b073a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:09.698511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fe6cdefe4174887b82a53df1e2a9fe9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:09.699176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.699462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.699737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.710608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:09.712084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dad681-9b1d-4f3c-b03c-6844e8ad0712 service glance] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:09.723406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1173/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:41:09.789438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a88ad6-df0e-456f-967d-55d6118c7235 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.789728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a88ad6-df0e-456f-967d-55d6118c7235 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.790026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a88ad6-df0e-456f-967d-55d6118c7235 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.792175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1205/2378] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:41:09.796927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b93e-29d7-4bd8-a96b-0c69ab70f401 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.797239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b93e-29d7-4bd8-a96b-0c69ab70f401 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.797512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b93e-29d7-4bd8-a96b-0c69ab70f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.816650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6840b93e-29d7-4bd8-a96b-0c69ab70f401 None None] MFA Rules not processed for user `2dd52fd2b7054851b62ae5b1d4f5ef29`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:09.817875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6840b93e-29d7-4bd8-a96b-0c69ab70f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:09.824456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1174/2379] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:41:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:09.837536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a88ad6-df0e-456f-967d-55d6118c7235 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:09.849322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a88ad6-df0e-456f-967d-55d6118c7235 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:09.853192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:09.854486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.854855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.855186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.862935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:09.864405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed72375-0107-4f35-b87e-f90587d2685c service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:09.875578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1206/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:09.968312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d79aa-3abb-41fc-bc97-ed5b1baedcca None None] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.968682 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d79aa-3abb-41fc-bc97-ed5b1baedcca None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.969043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d79aa-3abb-41fc-bc97-ed5b1baedcca None None] PATH_INFO: `/` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.970506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1175/2381] 10.222.0.33 () {54 vars in 921 bytes} [Wed Jun 15 00:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 00:41:09.976947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6466f4-1a06-43f9-84b6-4d9e9d0b2e1b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:09.977259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6466f4-1a06-43f9-84b6-4d9e9d0b2e1b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:09.977534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6466f4-1a06-43f9-84b6-4d9e9d0b2e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:09.995695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-af6466f4-1a06-43f9-84b6-4d9e9d0b2e1b None None] MFA Rules not processed for user `42f57540b6cd47559628031b441a1eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:09.996845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-af6466f4-1a06-43f9-84b6-4d9e9d0b2e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:10.006336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1207/2382] 10.222.0.33 () {58 vars in 1018 bytes} [Wed Jun 15 00:41:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:10.017189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d79aa-3abb-41fc-bc97-ed5b1baedcca None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:10.028716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d79aa-3abb-41fc-bc97-ed5b1baedcca None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:10.032752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:10.033394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:10.033677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:10.033950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:10.045958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:10.047449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:10.058942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1176/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:11.889796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bd7d7-e720-4802-bb02-f44145b13162 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:11.890326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bd7d7-e720-4802-bb02-f44145b13162 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:11.890422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bd7d7-e720-4802-bb02-f44145b13162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:11.908961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-677bd7d7-e720-4802-bb02-f44145b13162 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:11.910157 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-677bd7d7-e720-4802-bb02-f44145b13162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:11.921340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1208/2384] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:41:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:11.944981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:11.959403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275a023-f69d-47d3-9fa9-fbfde404d3e7 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:11.973053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:11.973700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:11.973979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:11.974252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:11.984021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:11.985852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:11.996812 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1177/2385] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:41:21.903774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=43 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:21.993018 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=41 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:28.165138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:28.165623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:28.165716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:28.183614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:28.184799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:28.192242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1209/2386] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:41:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:28.204576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:28.214946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9bd87-ae2d-4493-968d-1d8e392e3395 service swift] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:28.220863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:28.221920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:28.222221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:28.222494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:28.231160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:28.233616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d9aca-f1e9-4c79-b804-4ba0504f8eaf service swift] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:28.242075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1178/2387] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:41:29.630556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:29.700091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926b37-f95a-4e02-8be8-832e1ea2b83b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:29.706021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:29.707080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:29.707547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:29.707953 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] PATH_INFO: `/v3/users/cf48ed606dfa4e20adca3372c10b97cb` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:29.709376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf48ed606dfa4e20adca3372c10b97cb)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:29.711503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:29.765368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf48ed606dfa4e20adca3372c10b97cb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:29.766159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf48ed606dfa4e20adca3372c10b97cb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:29.777844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf48ed606dfa4e20adca3372c10b97cb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:29.786567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf48ed606dfa4e20adca3372c10b97cb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:29.802188 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf48ed606dfa4e20adca3372c10b97cb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:29.812171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1210/2388] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:41:29 2022] DELETE /identity/v3/users/cf48ed606dfa4e20adca3372c10b97cb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:41:29.816938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:29.817418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:29.817525 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:29.838829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:29.840169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:29.913438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1179/2389] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:41:29.923680 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:29.925363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:29.986092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a95473-2236-41cf-8e08-d099fca308d1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:29.989754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:29.990394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:29.990676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:29.990976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.000545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.001911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9960c2-e104-4c25-81d1-0a9c7f3e489a service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.011730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1211/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:30.286531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.287962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:30.300079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3c8b-1764-4c19-9abb-44e2b302f0bd None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.304647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.306821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.306821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.306821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] PATH_INFO: `/v3/projects/e44c61e61323479fbabcbc28a5b29995` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.307259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44c61e61323479fbabcbc28a5b29995)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.309199 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.358383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44c61e61323479fbabcbc28a5b29995', 'request_id': 'req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:30.363845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44c61e61323479fbabcbc28a5b29995', 'request_id': 'req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:30.371844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44c61e61323479fbabcbc28a5b29995', 'request_id': 'req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:30.382853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] The token cache is being invalidate because project e44c61e61323479fbabcbc28a5b29995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:41:30.383142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:30.383761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-0f887edb-4a22-415e-bcd9-b169ad2ff5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:30.385294 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1180/2391] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:41:30 2022] DELETE /identity/v3/projects/e44c61e61323479fbabcbc28a5b29995 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:41:30.390640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.390958 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.391232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.408923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:30.410115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:30.419527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:30.473800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1212/2392] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:30.482466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.482753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.483074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.501955 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:30.503193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:30.511099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1181/2393] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:30.515777 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.525965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084430b9-2e70-4851-8a83-76858bb876b0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.531174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.531822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.532130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.532401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.533124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:41:30.533427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.535038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.548183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1213/2394] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:41:30.552625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.562935 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59b6f0-097b-47a0-8c7e-c7291485a513 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.567733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.568343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.568622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.568894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.569587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.571564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.607713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1182/2395] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:41:30 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:41:30.617000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.628204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a45198-93cb-43e2-9f5d-9afb160cb40f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.685225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.685935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.686148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.686422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.687132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.689099 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.707740 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1214/2396] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:41:30 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:41:30.711761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.722065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e00b9-10be-4f03-9c53-39f0d2249ad5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.726821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.727462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.727741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.728010 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.728730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.730309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.740017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1183/2397] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:41:30.743849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.754385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35b1b-a0da-488f-aaf0-c9efc2cc8c28 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.758996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.759648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.759926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.760197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.760919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.762554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9867c-4e2b-4084-a693-f5625f5da2aa admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.773436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1215/2398] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:41:30.777293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.787009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca40f-7d0c-4274-a206-6f4e46986b40 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:30.791570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:30.792420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.792699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.792975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] PATH_INFO: `/v3/projects/765342b3db8c4b3bb31c8cccc0b5f8e7/users/8370996a3a8e4f19bda4e32ace40cca1/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.807686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=765342b3db8c4b3bb31c8cccc0b5f8e7, user_id=8370996a3a8e4f19bda4e32ace40cca1, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:30.810988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:30.818700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:30.825288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1184/2399] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:41:30 2022] PUT /identity/v3/projects/765342b3db8c4b3bb31c8cccc0b5f8e7/users/8370996a3a8e4f19bda4e32ace40cca1/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:41:30.835878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c431-4935-45e9-9514-6ff1f287de1c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:30.836192 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c431-4935-45e9-9514-6ff1f287de1c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:30.836469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c431-4935-45e9-9514-6ff1f287de1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:30.863618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-88a3c431-4935-45e9-9514-6ff1f287de1c None None] MFA Rules not processed for user `8370996a3a8e4f19bda4e32ace40cca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:30.864815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-88a3c431-4935-45e9-9514-6ff1f287de1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:30.927383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1216/2400] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:30.938526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:30.940328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:30.999705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fbaac-08cc-4ffc-8255-4412411fc49f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:31.003302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:31.003938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:31.004218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:31.004490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:31.012886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:31.014252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7cf2e-2264-4500-8a05-a3b0782e19cf service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:31.022951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1185/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:32.353405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:32.353899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:32.354014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:32.373350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] MFA Rules not processed for user `8370996a3a8e4f19bda4e32ace40cca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:32.374571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:32.381669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1217/2402] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:41.019059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=27 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:42.366269 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:49.662843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb4c-f8df-4e06-b846-cc1ba8174a7f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:49.663419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb4c-f8df-4e06-b846-cc1ba8174a7f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:49.663419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eebb4c-f8df-4e06-b846-cc1ba8174a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:49.682507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-c0eebb4c-f8df-4e06-b846-cc1ba8174a7f None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:49.683739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c0eebb4c-f8df-4e06-b846-cc1ba8174a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:49.741858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1186/2403] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:41:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:49.754930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:49.756369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:49.832801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8981fa-f85a-4d60-a6a2-e846e50594fa None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:49.837807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:49.838451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:49.838760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:49.839064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:49.847521 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:49.849111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:49.857675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1218/2404] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:41:50.357898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8af4-6162-4d8d-a1fd-80a0a773ae29 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:50.358191 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8af4-6162-4d8d-a1fd-80a0a773ae29 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:50.358472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f8af4-6162-4d8d-a1fd-80a0a773ae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:50.376215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-773f8af4-6162-4d8d-a1fd-80a0a773ae29 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:50.377398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-773f8af4-6162-4d8d-a1fd-80a0a773ae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:50.384565 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1187/2405] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:41:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:50.396916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:50.408226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577cdbc-2283-4114-a914-a2849ea7d4e9 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:50.414027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:50.414665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:50.414977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:50.415252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:50.424199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:50.425773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:50.434407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1219/2406] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:41:50.621240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c780a2-a51a-4b76-8674-e1125c6db9b3 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:50.621528 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c780a2-a51a-4b76-8674-e1125c6db9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:50.621803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c780a2-a51a-4b76-8674-e1125c6db9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:50.640721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f6c780a2-a51a-4b76-8674-e1125c6db9b3 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:50.641972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f6c780a2-a51a-4b76-8674-e1125c6db9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:50.648978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1188/2407] 10.222.0.33 () {58 vars in 1009 bytes} [Wed Jun 15 00:41:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:50.661834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:50.673056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd73a-3048-4d79-8ec6-8dd5b9643074 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:50.677994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:50.678619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:50.678935 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:50.679215 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:50.687254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:50.688823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:50.697047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1220/2408] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 00:41:51.016179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:51.016635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:51.016742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:51.034578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:51.035796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:51.042868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1189/2409] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:41:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:41:51.054564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:51.064970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7d6e-e89f-4bd5-94d2-fe60af35f50d service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:51.070037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:51.070672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:51.070985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:51.071265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:51.079735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:51.081308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc3586-cc47-4db4-a7a3-aa36d7f30d34 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:51.089297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1221/2410] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:41:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 00:41:53.177683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:53.245004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7550e-1d8d-4654-b05e-0f1fc26d5155 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:53.250416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:53.251473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:53.251758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:53.252033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] PATH_INFO: `/v3/users/8370996a3a8e4f19bda4e32ace40cca1` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:53.253240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8370996a3a8e4f19bda4e32ace40cca1)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:53.255198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:53.302078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8370996a3a8e4f19bda4e32ace40cca1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.302714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8370996a3a8e4f19bda4e32ace40cca1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.303329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8370996a3a8e4f19bda4e32ace40cca1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.312391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8370996a3a8e4f19bda4e32ace40cca1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.322243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8370996a3a8e4f19bda4e32ace40cca1'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.342448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1190/2411] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:41:53 2022] DELETE /identity/v3/users/8370996a3a8e4f19bda4e32ace40cca1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:41:53.347224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:53.348022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:53.348467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:53.375395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:41:53.376621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:53.434563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1222/2412] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:41:53.444509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:53.445903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:53.452647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:53.504842 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b332c6a-fdda-45f6-bd94-124b4dfe19e0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:53.508447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:53.509087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:53.509365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:53.509638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:53.518578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:53.520045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:53.529389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1191/2413] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:41:53.695747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:41:53.697542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:41:53.709528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54a93-a0af-4ce8-a125-c6e9309fba44 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:41:53.714352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:41:53.715258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:41:53.715534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:41:53.715833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] PATH_INFO: `/v3/projects/765342b3db8c4b3bb31c8cccc0b5f8e7` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:41:53.717510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765342b3db8c4b3bb31c8cccc0b5f8e7)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:41:53.719471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:41:53.766259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:41:53.769634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765342b3db8c4b3bb31c8cccc0b5f8e7', 'request_id': 'req-7d1bdcec-ca14-4800-aa1a-5932aa224510'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.775816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765342b3db8c4b3bb31c8cccc0b5f8e7', 'request_id': 'req-7d1bdcec-ca14-4800-aa1a-5932aa224510'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.783318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765342b3db8c4b3bb31c8cccc0b5f8e7', 'request_id': 'req-7d1bdcec-ca14-4800-aa1a-5932aa224510'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.794250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] The token cache is being invalidate because project 765342b3db8c4b3bb31c8cccc0b5f8e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:41:53.795162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.795798 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:41:53.797333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1223/2414] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:41:53 2022] DELETE /identity/v3/projects/765342b3db8c4b3bb31c8cccc0b5f8e7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:03.525214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=24 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:03.791298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:09.764225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:09.766080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:09.778039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eae817-4326-4b8c-b1c6-8478ac7f1814 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:09.835277 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:09.837045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:09.839276 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:09.840022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:09.840372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:09.841341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:09.843178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:09.853139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bdcec-ca14-4800-aa1a-5932aa224510 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:09.857256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:09.857938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:09.858222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:09.858623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:09.860651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:09.914632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:09.916179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:09.926586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1192/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:09.940879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:09.942408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:09.952598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1224/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:14.249073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:14.250823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:14.264055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f356d-667f-4893-970d-d1dea630857c service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:14.274149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:14.275488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:14.275889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:14.276440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] PATH_INFO: `/v3/users/db8e18e9735c4cd0afb79921383e8b67` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:14.279011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8e18e9735c4cd0afb79921383e8b67)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:14.280998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:14.338285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8e18e9735c4cd0afb79921383e8b67'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:14.339344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8e18e9735c4cd0afb79921383e8b67'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:14.340157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8e18e9735c4cd0afb79921383e8b67'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:14.352048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8e18e9735c4cd0afb79921383e8b67'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:14.364829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f7672b8d-a8ed-45c8-89fa-f404ad8648bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8e18e9735c4cd0afb79921383e8b67'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:14.394800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1193/2417] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:14 2022] DELETE /identity/v3/users/db8e18e9735c4cd0afb79921383e8b67 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:14.921203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:14.941874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c63128-8601-46cc-b6e4-bd7776ec63c2 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.027902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.029402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.029970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.030514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] PATH_INFO: `/v3/projects/ad57875727e24965a37faca9c9a30984` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.031984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad57875727e24965a37faca9c9a30984)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.034126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.092424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad57875727e24965a37faca9c9a30984', 'request_id': 'req-be6c75a1-bf9c-483d-9470-ed99b603f89a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.098642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad57875727e24965a37faca9c9a30984', 'request_id': 'req-be6c75a1-bf9c-483d-9470-ed99b603f89a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.108326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad57875727e24965a37faca9c9a30984', 'request_id': 'req-be6c75a1-bf9c-483d-9470-ed99b603f89a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.124526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] The token cache is being invalidate because project ad57875727e24965a37faca9c9a30984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:15.124817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.125659 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be6c75a1-bf9c-483d-9470-ed99b603f89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.130497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1225/2418] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:14 2022] DELETE /identity/v3/projects/ad57875727e24965a37faca9c9a30984 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:15.133074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.134001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.134312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.159282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:15.160859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:15.187653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:15.247489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1194/2419] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:15.249242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.249538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.249823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.277056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:15.279483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:15.292899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1226/2420] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:15.294375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.305596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4cdb7-c2d8-4f2b-9272-712932a1ad46 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.316076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.316756 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.317079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.317356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.318096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:15.318405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.320093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.339999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1195/2421] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:15.345147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.363827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89c658-b8f1-4f32-b938-1510adeb54b8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.372767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.373586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.373872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.374293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.375141 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.377704 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.399552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:15.416349 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1227/2422] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:15 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:15.421081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.432299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4161717-ef2e-4ff8-b6be-03ef45243f62 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.497810 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.498483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.498765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.499067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.499764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.501704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.521874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1196/2423] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:15 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:15.525931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.536520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ec246-0e7b-4b81-bde4-fb154e6a6575 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.541738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.542398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.542690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.543000 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.543766 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.546154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.557093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1228/2424] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:15.561479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.572715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259c16d-b415-45d0-8d53-0db46832fcc8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.580660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.581482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.581774 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.582120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.583485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.585304 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.599148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1197/2425] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:15.603630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.614877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77274982-01c7-4bf2-bfbb-a5833f1dbb47 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.620433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.621478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.622520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.623308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] PATH_INFO: `/v3/projects/bc9cd94ce22a4ed992a2632b1d080773/users/f08d7f1cd44d480d91c4869319aa701a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.631359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.634356 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:15.643905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9cd94ce22a4ed992a2632b1d080773, user_id=f08d7f1cd44d480d91c4869319aa701a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.646883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118de04f-28fe-46d8-ab58-26431aa68a6d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.653229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434904-8d8d-45d6-b297-399fc6899fcb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.659477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.661255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.661546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.661817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] PATH_INFO: `/v3/users/eceb9f042cb24343892f09454c7e685a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.663129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eceb9f042cb24343892f09454c7e685a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.665099 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.665222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1229/2426] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:15 2022] PUT /identity/v3/projects/bc9cd94ce22a4ed992a2632b1d080773/users/f08d7f1cd44d480d91c4869319aa701a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:15.676746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.677842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.678408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.723533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] MFA Rules not processed for user `f08d7f1cd44d480d91c4869319aa701a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:15.725036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:15.733209 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eceb9f042cb24343892f09454c7e685a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.733882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eceb9f042cb24343892f09454c7e685a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.734519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eceb9f042cb24343892f09454c7e685a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.741909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eceb9f042cb24343892f09454c7e685a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.750087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eceb9f042cb24343892f09454c7e685a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:15.764722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:15.771096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1198/2427] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:15 2022] DELETE /identity/v3/users/eceb9f042cb24343892f09454c7e685a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:15.800797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1230/2428] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:15.812514 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.813975 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:15.873730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f83df-8e78-467f-8308-8a3304fb2264 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:15.877439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:15.878114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:15.878393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:15.878666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:15.936521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:15.938175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349233ca-facc-4fb6-b5cd-e4b4f47758d6 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:15.948548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:15.950370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1199/2429] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:15.952493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:16.033326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55230c34-7e29-4904-83f1-f4dab6d16ac8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.085368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:16.093931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.095579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.095579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.095805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] PATH_INFO: `/v3/projects/32c13d8e3a764cf59f81e17eaca1c853` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.097259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c13d8e3a764cf59f81e17eaca1c853)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.099258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.148010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c13d8e3a764cf59f81e17eaca1c853', 'request_id': 'req-c4388112-f352-4756-9c66-6d64e5ced281'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:16.154923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c13d8e3a764cf59f81e17eaca1c853', 'request_id': 'req-c4388112-f352-4756-9c66-6d64e5ced281'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:16.162689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c13d8e3a764cf59f81e17eaca1c853', 'request_id': 'req-c4388112-f352-4756-9c66-6d64e5ced281'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:16.174412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] The token cache is being invalidate because project 32c13d8e3a764cf59f81e17eaca1c853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:16.174701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:16.175365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4388112-f352-4756-9c66-6d64e5ced281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:16.176919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1231/2430] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:15 2022] DELETE /identity/v3/projects/32c13d8e3a764cf59f81e17eaca1c853 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:16.182621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.182993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.183302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.213463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:16.214692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:16.286225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1200/2431] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:16.294894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.295182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.295459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.313856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:16.315390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:16.323226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1232/2432] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:16.327361 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.336784 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9a70a-ce3b-4d4c-8ce5-f751d8eb5728 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.341853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.342484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.342758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.343058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.343776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:16.344077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.345667 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.357609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1201/2433] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:16.361878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.371405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd508-2063-401c-a78b-111e796fcfa1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.377711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.378434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.378717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.379100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.379810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.381728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.406734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1233/2434] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:16 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:16.411248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.421757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fd8a-b787-4e7a-957e-140876beaf5b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.462290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:16.485119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.485777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.486060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.486335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.487220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.489146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.509894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1202/2435] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:16 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:16.514638 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.527321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18428a52-81ff-4832-b3e2-1217a3d4aa86 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.530400 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.531091 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.531372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.531647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.532374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.533997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.547973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1234/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:16.549248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.561886 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693ceec-0bc1-48b8-b84e-2535f78e8dfd admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.567704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.568365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.568648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.568648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.569785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.571268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ceb6-20d9-4026-b8fa-f37fc7663fc7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.583355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1203/2437] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:16.587283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.598063 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ef550-7cc8-499c-ad66-2307f606a1b9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.603286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.604162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.604446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.604721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] PATH_INFO: `/v3/projects/507242f1927b4f70bf73d23125dc6903/users/a3e3b8b59895461c8b9ec59e977aea5b/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.624055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=507242f1927b4f70bf73d23125dc6903, user_id=a3e3b8b59895461c8b9ec59e977aea5b, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.626835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.642750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1235/2438] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:16 2022] PUT /identity/v3/projects/507242f1927b4f70bf73d23125dc6903/users/a3e3b8b59895461c8b9ec59e977aea5b/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:16.652778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.653630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.653928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.682238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] MFA Rules not processed for user `a3e3b8b59895461c8b9ec59e977aea5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:16.683495 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:16.748051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1204/2439] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:16.760414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:16.762120 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:16.836621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dce01-c307-48b5-b6dd-8ddf559c5495 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:16.842156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:16.842822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:16.843113 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:16.843386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:16.851702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:16.853590 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:16.863816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1236/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:17.776502 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:17.778417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:17.834468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db41a3-b014-424b-b7a6-323a9062594b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:17.838193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:17.838859 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:17.839148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:17.839499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:17.841733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:17.903459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:17.904880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:17.909733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:17.914625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1205/2441] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:20.833267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:20.834965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:20.849532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134171a9-1738-4621-96f4-752f8ea2f0ec service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:20.854016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:20.854940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:20.855233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:20.855508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] PATH_INFO: `/v3/users/dc56e0e37b3d4445a69391142935abcf` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:20.856741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc56e0e37b3d4445a69391142935abcf)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:20.858660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:20.861542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:20.910058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc56e0e37b3d4445a69391142935abcf'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:20.910708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc56e0e37b3d4445a69391142935abcf'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:20.917993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc56e0e37b3d4445a69391142935abcf'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:20.925140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc56e0e37b3d4445a69391142935abcf'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:20.937662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-74353583-a1d7-480a-862a-8eda0d3f0cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc56e0e37b3d4445a69391142935abcf'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:20.943507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1237/2442] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:20 2022] DELETE /identity/v3/users/dc56e0e37b3d4445a69391142935abcf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:21.118860 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.130708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814bfe-5b93-46e5-9401-34411722632e service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.215881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.216788 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.217064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.217333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] PATH_INFO: `/v3/projects/47d48d22515e469fa53498b17ed80be7` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.218531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d48d22515e469fa53498b17ed80be7)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.220519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.272046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d48d22515e469fa53498b17ed80be7', 'request_id': 'req-8b5ae09f-dcf8-40e7-a303-c47727214da3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:21.278048 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d48d22515e469fa53498b17ed80be7', 'request_id': 'req-8b5ae09f-dcf8-40e7-a303-c47727214da3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:21.285966 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d48d22515e469fa53498b17ed80be7', 'request_id': 'req-8b5ae09f-dcf8-40e7-a303-c47727214da3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:21.299066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] The token cache is being invalidate because project 47d48d22515e469fa53498b17ed80be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:21.299355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:21.299992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8b5ae09f-dcf8-40e7-a303-c47727214da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:21.301551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1206/2443] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:21 2022] DELETE /identity/v3/projects/47d48d22515e469fa53498b17ed80be7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:21.306561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af1fe-b853-435b-8dac-b8971079427e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.306901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af1fe-b853-435b-8dac-b8971079427e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.307187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057af1fe-b853-435b-8dac-b8971079427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.325344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-057af1fe-b853-435b-8dac-b8971079427e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:21.326544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-057af1fe-b853-435b-8dac-b8971079427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:21.388924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1238/2444] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:21.397758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.398045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.398324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.418060 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:21.419946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:21.427871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1207/2445] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:21.432049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057af1fe-b853-435b-8dac-b8971079427e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.441603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057af1fe-b853-435b-8dac-b8971079427e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.446197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.446863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.447143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.447411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.448126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:21.448449 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.450038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.461244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1239/2446] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:21.465831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.475343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0953c8-b6a5-40a4-9ba6-f9018439589b None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.480017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.480659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.481482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.481757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.482454 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.484412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.512981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1208/2447] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:21.514181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.523991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7aeee-95fb-4884-874e-9f2c08682f30 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.584565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.585334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.585688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.586032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.586836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.588846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.604103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:21.614232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1240/2448] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:21.615616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.632084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fa528-8cc1-4171-80d4-150608bc60cd admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.637343 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.638003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.638293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.638560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.639349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.640936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.655411 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1209/2449] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:21.656582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.666989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf6e5c-2cec-445d-b517-a4c65f7d9b6b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.672230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.672891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.673177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.673457 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.674190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.675833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261ae97-4df5-4b1f-a3d0-7e03355bd2fd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.686448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1241/2450] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:21.690607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.704751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88f3e-2a2b-40cb-88a3-4f994718cc98 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.709653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.710577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.710843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.711133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] PATH_INFO: `/v3/projects/6806f7454efd426daae4af3e2d00f67f/users/26e5c6b997ec4df5a7614faadc94bb72/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.727351 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6806f7454efd426daae4af3e2d00f67f, user_id=26e5c6b997ec4df5a7614faadc94bb72, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.730074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.743817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1210/2451] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:21 2022] PUT /identity/v3/projects/6806f7454efd426daae4af3e2d00f67f/users/26e5c6b997ec4df5a7614faadc94bb72/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:21.754336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.754625 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.754934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.782248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] MFA Rules not processed for user `26e5c6b997ec4df5a7614faadc94bb72`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:21.783495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:21.853244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1242/2452] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:21.857676 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.891004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:21.925213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e544-7f3c-4516-b734-b590306254e7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:21.930072 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:21.930868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:21.931166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:21.931439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:21.932231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:21.934249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:21.962829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1211/2453] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:21.964961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:21.978842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309b585-5035-4c2e-8d98-c58f34c3e4f2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.039918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.041021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.041516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.041965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.043418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.046144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.066270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1243/2454] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:21 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:22.070128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.080727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0bb9-bf21-488e-864e-981a7c92c467 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.085384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.086025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.086305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.086573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.087320 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.088898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.100123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1212/2455] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:22.103997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.113873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfbfe5-1b23-4e09-89fc-21d27e290121 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.118433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.119338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.119617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.119896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] PATH_INFO: `/v3/projects/a6e561b991b64aeab561f4b1cdd70c58/users/e14cee0509ef45a697930a8f3bffa5e2/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.134940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e561b991b64aeab561f4b1cdd70c58, user_id=e14cee0509ef45a697930a8f3bffa5e2, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.138286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.151997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1244/2456] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:22 2022] PUT /identity/v3/projects/a6e561b991b64aeab561f4b1cdd70c58/users/e14cee0509ef45a697930a8f3bffa5e2/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:22.155852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.165893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ab88-6f22-452a-8b0a-3d60bb386e18 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.229978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.230659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.231032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.231273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.232011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.233622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c0f7f-3869-4e91-94f7-e17b548df6f3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.244313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1213/2457] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:22.249693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.264784 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00960-875d-4580-a413-fa01506d2d15 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.269552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.270438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.270718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.271021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] PATH_INFO: `/v3/projects/a6e561b991b64aeab561f4b1cdd70c58/users/e14cee0509ef45a697930a8f3bffa5e2/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.272999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e561b991b64aeab561f4b1cdd70c58, user_id=e14cee0509ef45a697930a8f3bffa5e2, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.275706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.287046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1245/2458] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:22 2022] PUT /identity/v3/projects/a6e561b991b64aeab561f4b1cdd70c58/users/e14cee0509ef45a697930a8f3bffa5e2/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:22.297201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.297493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.297766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.325496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] MFA Rules not processed for user `e14cee0509ef45a697930a8f3bffa5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:22.326622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:22.399894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1214/2459] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:22.411199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.412910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:22.447038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:22.481390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5736a5-5ae3-4d61-8b89-90d0626c0a6c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.485024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.485691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.485970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.486241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.495809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.497860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033fd7-0c71-46d4-ad5b-e01cef2701df service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:22.507272 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1246/2460] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:22.878901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:22.915503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:22.961291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e84a4-2b6a-4b43-9727-bb877c81d607 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:22.977457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:22.978362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:22.978690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:22.979047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] PATH_INFO: `/v3/users/26e5c6b997ec4df5a7614faadc94bb72` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:22.980598 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e5c6b997ec4df5a7614faadc94bb72)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:22.983270 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:23.042201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e5c6b997ec4df5a7614faadc94bb72'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.042957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e5c6b997ec4df5a7614faadc94bb72'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.044197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e5c6b997ec4df5a7614faadc94bb72'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.056622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e5c6b997ec4df5a7614faadc94bb72'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.064696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e5c6b997ec4df5a7614faadc94bb72'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.082745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1215/2461] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:22 2022] DELETE /identity/v3/users/26e5c6b997ec4df5a7614faadc94bb72 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:23.087427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:23.087928 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:23.088011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:23.107086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:23.108284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:23.173193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1247/2462] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:23.183105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:23.184668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:23.243511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299eef1-a30d-4b56-89ec-435686af636a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:23.247205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:23.247840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:23.248140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:23.248396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:23.257293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:23.258721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:23.268687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1216/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:23.545585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:23.547041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:23.564427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57135b-c453-48ac-9358-4b2f0d31d442 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:23.569087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:23.569951 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:23.570238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:23.570514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] PATH_INFO: `/v3/projects/6806f7454efd426daae4af3e2d00f67f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:23.571734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6806f7454efd426daae4af3e2d00f67f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:23.573689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:23.621012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6806f7454efd426daae4af3e2d00f67f', 'request_id': 'req-be3b73cf-d2bf-4fb5-8167-bc98200b111e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.627004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6806f7454efd426daae4af3e2d00f67f', 'request_id': 'req-be3b73cf-d2bf-4fb5-8167-bc98200b111e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.633783 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6806f7454efd426daae4af3e2d00f67f', 'request_id': 'req-be3b73cf-d2bf-4fb5-8167-bc98200b111e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.645393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] The token cache is being invalidate because project 6806f7454efd426daae4af3e2d00f67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:23.645700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.646364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.647911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1248/2464] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:23 2022] DELETE /identity/v3/projects/6806f7454efd426daae4af3e2d00f67f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:23.651843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:23.661987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc31532-a274-4d73-bf5e-e98445d5ae0a service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:23.703484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:23.720318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:23.721439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:23.721707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:23.721977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] PATH_INFO: `/v3/users/e14cee0509ef45a697930a8f3bffa5e2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:23.723186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14cee0509ef45a697930a8f3bffa5e2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:23.725123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:23.770366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14cee0509ef45a697930a8f3bffa5e2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.771098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14cee0509ef45a697930a8f3bffa5e2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.771710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14cee0509ef45a697930a8f3bffa5e2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.779193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14cee0509ef45a697930a8f3bffa5e2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.786034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14cee0509ef45a697930a8f3bffa5e2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:23.803834 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1217/2465] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:23 2022] DELETE /identity/v3/users/e14cee0509ef45a697930a8f3bffa5e2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:24.077274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.078717 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.094123 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b73cf-d2bf-4fb5-8167-bc98200b111e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.160570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:24.168664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.170144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.170575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.171530 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] PATH_INFO: `/v3/projects/a6e561b991b64aeab561f4b1cdd70c58` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.173502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e561b991b64aeab561f4b1cdd70c58)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.175863 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.195687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.195687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.213620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca9f9-a282-4b12-9b3a-7ab3278c97bf admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.219316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.220211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.220497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.221017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] PATH_INFO: `/v3/users/a3e3b8b59895461c8b9ec59e977aea5b` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.222229 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e3b8b59895461c8b9ec59e977aea5b)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.224374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.249939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e561b991b64aeab561f4b1cdd70c58', 'request_id': 'req-94deb227-6274-465b-9c35-992ed9c610d1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.257115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e561b991b64aeab561f4b1cdd70c58', 'request_id': 'req-94deb227-6274-465b-9c35-992ed9c610d1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.265367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e561b991b64aeab561f4b1cdd70c58', 'request_id': 'req-94deb227-6274-465b-9c35-992ed9c610d1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.278834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] The token cache is being invalidate because project a6e561b991b64aeab561f4b1cdd70c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:24.279137 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.279815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94deb227-6274-465b-9c35-992ed9c610d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.281545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1249/2466] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:24 2022] DELETE /identity/v3/projects/a6e561b991b64aeab561f4b1cdd70c58 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:24.283141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e3b8b59895461c8b9ec59e977aea5b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.283917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e3b8b59895461c8b9ec59e977aea5b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.284547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e3b8b59895461c8b9ec59e977aea5b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.286303 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a75-1912-4f61-8dd5-c6c7d425aaa1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.286727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a75-1912-4f61-8dd5-c6c7d425aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.286997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a75-1912-4f61-8dd5-c6c7d425aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.293248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e3b8b59895461c8b9ec59e977aea5b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.304977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c551dc79-d44c-49c8-8347-f9308989ab48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e3b8b59895461c8b9ec59e977aea5b'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.305669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a0393a75-1912-4f61-8dd5-c6c7d425aaa1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:24.306924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a0393a75-1912-4f61-8dd5-c6c7d425aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.327258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1218/2467] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:24 2022] DELETE /identity/v3/users/a3e3b8b59895461c8b9ec59e977aea5b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:24.331662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.332232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.332513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.350443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:24.351765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.372972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1250/2468] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:24.382374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.382748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.383075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.388217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:24.401963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:24.403245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.420702 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1219/2469] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:24.430582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.432023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.465613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1251/2470] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:24.469767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.479591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdadc3-fdd1-4e4f-9e5c-e78e3d95e985 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.485398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.486270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.486585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.486894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.487712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:24.488049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.489799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.493255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c47c9-6277-42aa-8b1e-4b8d4f02584e None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.498214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.498894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.499176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.499986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.506369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1252/2471] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:42:24.508019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.509971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.511492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.518314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25ff9-4919-484e-bf02-29887d781f3c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.522824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1220/2472] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:24.527472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.528201 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.528500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.528795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.529497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.531474 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.543010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:24.566871 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1253/2473] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:24.568081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.579811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236a98d-56ea-49de-a0c9-3b2a44d4628e service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.644433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.645186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.645467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.645739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.646461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.648611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.669802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1221/2474] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:24.673683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.686990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816863f-9c99-461d-b91a-8cc040a3a049 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.692193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.692861 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.693150 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.693424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.694155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.695802 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.710154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1254/2475] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:24.711369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.721503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb17a3-71f3-464f-b7fb-aac338f71c02 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.726131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.726811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.727102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.727371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.728120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.729695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.740223 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1222/2476] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:24.744380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.754991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b657e-7b35-413b-b903-6bd7b70fcfeb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.759748 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.760689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.761036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.761375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] PATH_INFO: `/v3/projects/36d48f1860604b60bc9559eabc1a4aaa/users/dde44eef709446ce80278f4a6c971f26/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.777105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d48f1860604b60bc9559eabc1a4aaa, user_id=dde44eef709446ce80278f4a6c971f26, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.779932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908db96-597b-4c6c-b308-180f6f5bf446 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.796868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1255/2477] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:24 2022] PUT /identity/v3/projects/36d48f1860604b60bc9559eabc1a4aaa/users/dde44eef709446ce80278f4a6c971f26/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:24.803961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.805889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.807756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.808014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.808204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.821141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c082-805e-4ce8-90f9-057054976c11 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.842332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] MFA Rules not processed for user `dde44eef709446ce80278f4a6c971f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:24.843536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:24.875234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:24.882406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.883327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.884097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.884380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] PATH_INFO: `/v3/projects/507242f1927b4f70bf73d23125dc6903` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.885583 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=507242f1927b4f70bf73d23125dc6903)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.887632 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.909927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1256/2478] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:24.914331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:24.926093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216fb8b-cd1c-4c67-8d71-16ff2bccc779 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:24.941114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '507242f1927b4f70bf73d23125dc6903', 'request_id': 'req-67727880-1bbf-4979-aea0-24c7fc9bce5f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.946551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '507242f1927b4f70bf73d23125dc6903', 'request_id': 'req-67727880-1bbf-4979-aea0-24c7fc9bce5f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.953921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '507242f1927b4f70bf73d23125dc6903', 'request_id': 'req-67727880-1bbf-4979-aea0-24c7fc9bce5f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.965484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] The token cache is being invalidate because project 507242f1927b4f70bf73d23125dc6903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:24.965772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.966432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-67727880-1bbf-4979-aea0-24c7fc9bce5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:24.968044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1223/2479] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:24 2022] DELETE /identity/v3/projects/507242f1927b4f70bf73d23125dc6903 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:24.972840 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c35-10b0-4751-9381-ab5bdae36d6b None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.973110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c35-10b0-4751-9381-ab5bdae36d6b None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.973384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c35-10b0-4751-9381-ab5bdae36d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.986971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:24.987626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:24.987902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:24.988175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:24.988864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:24.990988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:24.991877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-99172c35-10b0-4751-9381-ab5bdae36d6b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:24.993132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-99172c35-10b0-4751-9381-ab5bdae36d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.003309 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1224/2480] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:25.012517 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.012857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.013081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.016745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1257/2481] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:24 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:25.020724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.022376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.031078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:25.032241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.056384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:25.089839 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56d21f-3320-4053-a1d9-add86097f493 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.093391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1225/2482] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:25.094709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.095406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.095697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.096228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.096996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.097827 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.098979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.107453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4538e-c230-4732-8b64-f178e5c86b68 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.112831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.113474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.113797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.114077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.114829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:25.115144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.116746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.119434 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1258/2483] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:25.123743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.129523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1226/2484] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:42:25.134423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.134592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121735fe-fba4-4586-bcf5-14fc63bea008 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.139267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.139968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.140253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.140529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.141255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.142878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.144735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f909b-b50c-4b4e-b9f0-83b342225fd4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.151023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.151654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.151929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.152231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.153450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.155389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.161026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1259/2485] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:25.161994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.173159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb213afb-1d53-4080-9eb0-725dc01a6328 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.179005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.179949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.180328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.180698 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] PATH_INFO: `/v3/projects/e0dd210cd2e24651977a27631091a1ac/users/1b4c32c34fd74129a75716cef458e3e0/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.187092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1227/2486] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:25.188084 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.196812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0dd210cd2e24651977a27631091a1ac, user_id=1b4c32c34fd74129a75716cef458e3e0, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.199557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.212497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1260/2487] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:25 2022] PUT /identity/v3/projects/e0dd210cd2e24651977a27631091a1ac/users/1b4c32c34fd74129a75716cef458e3e0/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:25.216869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.239938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:25.256201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d31176-4187-4fd6-a56f-a78af1915213 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.280394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9742-2a95-4409-938e-13cf7d887bcf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.285143 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.285801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.286081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.286433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.287125 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.288723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.302318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1261/2488] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:25.304057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.312222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.312889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.313169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.313442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.313577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5cb3-6c8c-47f4-8e20-dfba90355bfa admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.314131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.316126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.318425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.319359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.319640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.319914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] PATH_INFO: `/v3/projects/e0dd210cd2e24651977a27631091a1ac/users/1b4c32c34fd74129a75716cef458e3e0/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.321884 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0dd210cd2e24651977a27631091a1ac, user_id=1b4c32c34fd74129a75716cef458e3e0, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.327306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7f8a4-2ba2-43ed-b4a3-d71caa7577ab admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.337762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1228/2489] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:25.339032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1262/2490] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:25 2022] PUT /identity/v3/projects/e0dd210cd2e24651977a27631091a1ac/users/1b4c32c34fd74129a75716cef458e3e0/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:25.348097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.352803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.353210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.353526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.363946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8718b55-f06e-4943-82c3-415069f2c02c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.388912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] MFA Rules not processed for user `1b4c32c34fd74129a75716cef458e3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:25.390072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.440484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.441189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.441544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.441770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.442504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.444176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.455891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1229/2491] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:25.459786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.470379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86031c35-7914-476b-a6c4-2e316f7f466f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.476567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.477295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.477638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.477971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.478766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.480506 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.491957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1230/2492] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:25.495488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:25.497244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.502891 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1263/2493] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:25.507477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c278f-9b20-4c66-b735-c82aa1561035 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.513080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.513957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.514237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.514510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] PATH_INFO: `/v3/projects/5da5cde44f9f4add8cd3c4217b1a07de/users/c9522b96c1974b2f88c05b47525527d7/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.517919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.519372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.530724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da5cde44f9f4add8cd3c4217b1a07de, user_id=c9522b96c1974b2f88c05b47525527d7, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.533432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0c3ee-ac14-4714-ad8c-01e89a6a11d8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.546577 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1231/2494] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:25 2022] PUT /identity/v3/projects/5da5cde44f9f4add8cd3c4217b1a07de/users/c9522b96c1974b2f88c05b47525527d7/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:25.556500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.556794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.557079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.582520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877263e1-583a-46ea-9f91-e794899a25ee None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.586197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] MFA Rules not processed for user `c9522b96c1974b2f88c05b47525527d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:25.587512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:25.624338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:25.635561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.636320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.636603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.637414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.660939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1232/2495] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:25.665797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.705191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.706645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.717669 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1264/2496] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:25.732641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394efc-fb9f-448e-8353-d847cbfc9758 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.737754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.738408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.738699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.739006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.739697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.741610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.770612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1233/2497] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:25.771705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.782265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e9c64-832a-4829-ba65-88b0068a4db8 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.849439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.850115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.850383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.850682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.851409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.853325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.860413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:25.875895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1265/2498] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:25 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:25.879741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.890331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0735f-08b9-4a4b-bc98-b54f0815a5aa admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.895467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.896154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.896520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.896710 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.897622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.899652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.914768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1234/2499] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:25.916045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.927568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70e41-f606-47f4-bcbc-fd04b44db732 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:25.933298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:25.934326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:25.934665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:25.935028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] PATH_INFO: `/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a/users/1ac0016233d046309bed4f91991d631d/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:25.951885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9474f3e982ee4c019bf5cd36a8de8b8a, user_id=1ac0016233d046309bed4f91991d631d, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:25.954643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:25.968922 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1266/2500] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:25 2022] PUT /identity/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a/users/1ac0016233d046309bed4f91991d631d/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:25.973278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:25.983997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6047cb-efef-4139-a5bf-7c19ad439f72 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.034992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.047029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.047695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.047979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.048291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.049052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.050647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.064874 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1235/2501] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:26.065868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.076126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79221c-a478-4599-82eb-dbadcad5d984 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.081192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.082037 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.082317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.082590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] PATH_INFO: `/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a/users/1ac0016233d046309bed4f91991d631d/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.084591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9474f3e982ee4c019bf5cd36a8de8b8a, user_id=1ac0016233d046309bed4f91991d631d, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.087226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5fd14-e64d-4cf9-b5ed-2f0e89fcd405 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.098299 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1236/2502] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:26 2022] PUT /identity/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a/users/1ac0016233d046309bed4f91991d631d/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:26.101985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de09c0-811d-4d18-b000-94c6e677160b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.108543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.108832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.109109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.135173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] MFA Rules not processed for user `1ac0016233d046309bed4f91991d631d`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:26.136375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.155291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:26.161365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4c32c34fd74129a75716cef458e3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0dd210cd2e24651977a27631091a1ac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.162046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.162333 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.162612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.163416 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:26.163736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.165492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.177691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1267/2503] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:42:26.181745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.192153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d413b72-72f4-4882-8937-701f83b01865 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.197878 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4c32c34fd74129a75716cef458e3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0dd210cd2e24651977a27631091a1ac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.198527 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.199358 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.200295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.201049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.203026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.209346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1237/2504] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:26.222595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.224450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.243968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1268/2505] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:26 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:26.247917 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.286561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a5f6b-d275-402d-b5c6-eed2b0ea8ed1 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.300492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:26.319078 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f524f2f-8647-4c7b-b1ff-19fbcb3a2254 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.323763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4c32c34fd74129a75716cef458e3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0dd210cd2e24651977a27631091a1ac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.324967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.325250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.325524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.326253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:26.326562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.328203 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.335210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.335852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.336131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.336409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.339582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1269/2506] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:42:26.376892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.387944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c784169-9cce-4737-b211-1bdf41c87b23 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.392771 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4c32c34fd74129a75716cef458e3e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0dd210cd2e24651977a27631091a1ac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.393631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.393910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.394182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] PATH_INFO: `/v3/projects/aed9866a115f4f6cb20f90aac1a8de8c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.395393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aed9866a115f4f6cb20f90aac1a8de8c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.398018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.399790 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.401220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.411265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1238/2507] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:26.449149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed9866a115f4f6cb20f90aac1a8de8c', 'request_id': 'req-62530299-b5fa-436a-998c-9556461e5296'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.455229 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed9866a115f4f6cb20f90aac1a8de8c', 'request_id': 'req-62530299-b5fa-436a-998c-9556461e5296'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.462592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed9866a115f4f6cb20f90aac1a8de8c', 'request_id': 'req-62530299-b5fa-436a-998c-9556461e5296'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.475260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] The token cache is being invalidate because project aed9866a115f4f6cb20f90aac1a8de8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:26.475547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.476214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.477983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1270/2508] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:26 2022] DELETE /identity/v3/projects/aed9866a115f4f6cb20f90aac1a8de8c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:26.532387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.534088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.571360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:26.628130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16918c8-5bbe-481c-8486-9c06e4d27449 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.700404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.700775 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.701056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.701290 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] PATH_INFO: `/v3/users/f08d7f1cd44d480d91c4869319aa701a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.702596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08d7f1cd44d480d91c4869319aa701a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.704648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.720036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.721693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.735649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62530299-b5fa-436a-998c-9556461e5296 tempest-VolumeQuotaClassesTest-1734730595 tempest-VolumeQuotaClassesTest-1734730595-project-admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.742442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.743357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.743708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.743909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] PATH_INFO: `/v3/users/dde44eef709446ce80278f4a6c971f26` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.745469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde44eef709446ce80278f4a6c971f26)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.748230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:26.767362 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08d7f1cd44d480d91c4869319aa701a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.768052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08d7f1cd44d480d91c4869319aa701a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.768698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08d7f1cd44d480d91c4869319aa701a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.776430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08d7f1cd44d480d91c4869319aa701a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.785958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-f3f38817-fc93-427b-b447-7aae8b3215c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08d7f1cd44d480d91c4869319aa701a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.805453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1239/2509] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:26 2022] DELETE /identity/v3/users/f08d7f1cd44d480d91c4869319aa701a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:26.805781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde44eef709446ce80278f4a6c971f26'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.806375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde44eef709446ce80278f4a6c971f26'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.810330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.810633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.811038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.816571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde44eef709446ce80278f4a6c971f26'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.825425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde44eef709446ce80278f4a6c971f26'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.830755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:26.831939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.841803 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f1690937-b626-4ef5-b7a3-d1281592adae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde44eef709446ce80278f4a6c971f26'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:26.849315 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1271/2510] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:26 2022] DELETE /identity/v3/users/dde44eef709446ce80278f4a6c971f26 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:26.855929 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.856296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.856639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.880348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:26.883050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.892130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:26.909359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1240/2511] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:26.910186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.911787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.919466 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:26.927211 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1f8f-e1f2-404d-bd5f-6a53072ac336 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:26.955974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1272/2512] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:26.957163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:26.958408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:26.990443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:26.991359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:26.991637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:26.991914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] PATH_INFO: `/v3/users/c9522b96c1974b2f88c05b47525527d7` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:26.993111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9522b96c1974b2f88c05b47525527d7)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:26.995087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.018706 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f966de2-7517-4028-a05c-35f466bf8bf7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.023266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.023909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.024247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.024512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.034619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.036074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.042579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9522b96c1974b2f88c05b47525527d7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.043801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9522b96c1974b2f88c05b47525527d7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.044429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9522b96c1974b2f88c05b47525527d7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.045914 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1273/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:27.046690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.055133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9146b9-2dcc-4e1e-939c-3f9db1fc18c8 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.057558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9522b96c1974b2f88c05b47525527d7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.060210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.061061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.061351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.061759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.063867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.066733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9522b96c1974b2f88c05b47525527d7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.077275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.078700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4c274-c151-436a-937d-2a70a470dcf9 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.087732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1241/2514] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:26 2022] DELETE /identity/v3/users/c9522b96c1974b2f88c05b47525527d7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:27.089296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1274/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:27.092193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.092481 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.092754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.111339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:27.112554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.173903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1275/2516] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:27.184467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.186182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.251825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09604d-45a6-4387-8ebd-ff81634fd5e3 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.255653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.256301 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.256584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.256865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.268515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.269960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.280441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1242/2517] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:27.617547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.619206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.633606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9d00-213f-46b3-a3dd-501d7c44e99c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.639563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.640640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.641062 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.641540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] PATH_INFO: `/v3/projects/bc9cd94ce22a4ed992a2632b1d080773` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.642727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9cd94ce22a4ed992a2632b1d080773)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.644677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.677068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.678492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.696448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10f15c-24c5-40a3-873d-25096d846575 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.711143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:27.724819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9cd94ce22a4ed992a2632b1d080773', 'request_id': 'req-22168455-0479-45da-a4cd-3caa6b8d3ea2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.733444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9cd94ce22a4ed992a2632b1d080773', 'request_id': 'req-22168455-0479-45da-a4cd-3caa6b8d3ea2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.735719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:27.745614 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9cd94ce22a4ed992a2632b1d080773', 'request_id': 'req-22168455-0479-45da-a4cd-3caa6b8d3ea2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.760469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] The token cache is being invalidate because project bc9cd94ce22a4ed992a2632b1d080773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:27.760687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.761403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22168455-0479-45da-a4cd-3caa6b8d3ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.764054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1276/2518] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:27 2022] DELETE /identity/v3/projects/bc9cd94ce22a4ed992a2632b1d080773 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:27.768644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.768963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.769442 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.780981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.781884 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.782162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.782433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] PATH_INFO: `/v3/projects/36d48f1860604b60bc9559eabc1a4aaa` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.783721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d48f1860604b60bc9559eabc1a4aaa)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.785751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.794285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:27.795723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.805144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1277/2519] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:27.811864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.813676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.830433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87de379-fbeb-485f-85d1-9576a28c12ed None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.847448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d48f1860604b60bc9559eabc1a4aaa', 'request_id': 'req-519da773-267a-49e5-abf7-2485c7f1260d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.853121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d48f1860604b60bc9559eabc1a4aaa', 'request_id': 'req-519da773-267a-49e5-abf7-2485c7f1260d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.864256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d48f1860604b60bc9559eabc1a4aaa', 'request_id': 'req-519da773-267a-49e5-abf7-2485c7f1260d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.876217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] The token cache is being invalidate because project 36d48f1860604b60bc9559eabc1a4aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:27.876507 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.877156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-519da773-267a-49e5-abf7-2485c7f1260d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.881433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1243/2520] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:27 2022] DELETE /identity/v3/projects/36d48f1860604b60bc9559eabc1a4aaa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:27.881433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.881433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.881433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.898758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:27.900020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.904944 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.905840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.906145 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.906420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] PATH_INFO: `/v3/projects/5da5cde44f9f4add8cd3c4217b1a07de` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.907630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da5cde44f9f4add8cd3c4217b1a07de)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.908665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1244/2521] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:27.909500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.909602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.910707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.922374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee3028-64b5-4ad2-be2e-900545864acc None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:27.927049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:27.927892 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:27.928175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:27.928448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] PATH_INFO: `/v3/users/1b4c32c34fd74129a75716cef458e3e0` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:27.929634 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4c32c34fd74129a75716cef458e3e0)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:27.931613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:27.957425 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da5cde44f9f4add8cd3c4217b1a07de', 'request_id': 'req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.963657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da5cde44f9f4add8cd3c4217b1a07de', 'request_id': 'req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.972152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da5cde44f9f4add8cd3c4217b1a07de', 'request_id': 'req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.981536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4c32c34fd74129a75716cef458e3e0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.982186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4c32c34fd74129a75716cef458e3e0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.982853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4c32c34fd74129a75716cef458e3e0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.983751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] The token cache is being invalidate because project 5da5cde44f9f4add8cd3c4217b1a07de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:27.984035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.984712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.986316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1278/2522] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:27 2022] DELETE /identity/v3/projects/5da5cde44f9f4add8cd3c4217b1a07de => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:27.987111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:27.988868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:27.992705 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4c32c34fd74129a75716cef458e3e0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:27.999781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4c32c34fd74129a75716cef458e3e0'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.017576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1245/2523] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:27 2022] DELETE /identity/v3/users/1b4c32c34fd74129a75716cef458e3e0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:28.018283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.019716 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.030867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.064957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee8ac8-140e-48c7-ab50-55579dc8f7d8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.080728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.098489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a93b0-bc16-4089-8c09-37c9b7326f1f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.103887 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.105168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.105168 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.105556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] PATH_INFO: `/v3/users/1ac0016233d046309bed4f91991d631d` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.106878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac0016233d046309bed4f91991d631d)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.108888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.140423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.141279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.141562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.141832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.142691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:28.143035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.144727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.157378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1279/2524] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:28.161276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.167313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac0016233d046309bed4f91991d631d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.168387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac0016233d046309bed4f91991d631d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.169044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac0016233d046309bed4f91991d631d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.178879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac0016233d046309bed4f91991d631d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.182198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195da5-e8d4-43f6-93a2-df1ad81ffa7d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.188428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac0016233d046309bed4f91991d631d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.189068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.189837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.190265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.191158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.192491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.194074 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.212830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1246/2525] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:28 2022] DELETE /identity/v3/users/1ac0016233d046309bed4f91991d631d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:28.226819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1280/2526] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:28.228949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.237500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.314648 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4317-816e-4873-8920-1eaa89ce7df5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.319711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.321925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.359505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.396032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.396795 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.397357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.397491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.398232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.400347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.406005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d850a-64eb-4afe-ba4c-739c9f5a1b92 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.412339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.413553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.414029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.414355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] PATH_INFO: `/v3/projects/e0dd210cd2e24651977a27631091a1ac` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.416069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0dd210cd2e24651977a27631091a1ac)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.418251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.423237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1247/2527] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/users => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:28.427349 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.439694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eec8-14e8-4f50-82e8-e9d674efe51b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.444640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.445380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.445718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.446062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.446900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.448533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.462808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1248/2528] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:28.466670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.478974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf4754-5210-4495-806e-a406b83c7220 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.479696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0dd210cd2e24651977a27631091a1ac', 'request_id': 'req-10481c05-ae96-42e4-88d9-ac67cf275b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.485809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0dd210cd2e24651977a27631091a1ac', 'request_id': 'req-10481c05-ae96-42e4-88d9-ac67cf275b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.503414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0dd210cd2e24651977a27631091a1ac', 'request_id': 'req-10481c05-ae96-42e4-88d9-ac67cf275b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.516966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] The token cache is being invalidate because project e0dd210cd2e24651977a27631091a1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:28.517258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.518094 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-10481c05-ae96-42e4-88d9-ac67cf275b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.519743 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1281/2529] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:28 2022] DELETE /identity/v3/projects/e0dd210cd2e24651977a27631091a1ac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:28.524301 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.524904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.525283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.544912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:28.546311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.549600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.550504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.550917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.551284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.551930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.553693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716e400-9a90-4750-921e-a1a258b1b4f5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.556006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1282/2530] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:28.563494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.564870 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.566066 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1249/2531] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:28.568558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.569020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.569433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.577411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60957-7af1-4cec-b7fd-ec3d35b7698e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.582495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.583390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.583670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.583940 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] PATH_INFO: `/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.585226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9474f3e982ee4c019bf5cd36a8de8b8a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.587180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.593715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:28.595633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.597183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.605206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1250/2532] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:28.606286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.607572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.619811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f434f3-ea43-4339-9560-01f7e8a59884 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.625105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.625972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.626277 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.626545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] PATH_INFO: `/v3/projects/dff10b2d0ea3435a8848e73f82c59a1e/users/b19ae1793f01402fa3027431791219d6/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.640104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9474f3e982ee4c019bf5cd36a8de8b8a', 'request_id': 'req-6e98ba27-4aa0-428e-8719-a5b348402b81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.641721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff10b2d0ea3435a8848e73f82c59a1e, user_id=b19ae1793f01402fa3027431791219d6, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.644435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.645811 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9474f3e982ee4c019bf5cd36a8de8b8a', 'request_id': 'req-6e98ba27-4aa0-428e-8719-a5b348402b81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.653693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9474f3e982ee4c019bf5cd36a8de8b8a', 'request_id': 'req-6e98ba27-4aa0-428e-8719-a5b348402b81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.657873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1251/2533] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:28 2022] PUT /identity/v3/projects/dff10b2d0ea3435a8848e73f82c59a1e/users/b19ae1793f01402fa3027431791219d6/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:28.659595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.665092 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] The token cache is being invalidate because project 9474f3e982ee4c019bf5cd36a8de8b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:28.665374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.666036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6e98ba27-4aa0-428e-8719-a5b348402b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:28.668682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1283/2534] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:28 2022] DELETE /identity/v3/projects/9474f3e982ee4c019bf5cd36a8de8b8a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:28.671072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.671072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.671072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.697289 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] MFA Rules not processed for user `b19ae1793f01402fa3027431791219d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:28.698540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.726143 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1755-cac5-4086-ad04-e8ea73b61e8a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.731374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.732023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.732306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.732621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.733283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:28.733584 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.735216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40db7f5-e366-4396-9d74-b8b3bb761504 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.747613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1252/2535] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:28.748846 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa70136-7690-4708-8d09-49271757947a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.749187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa70136-7690-4708-8d09-49271757947a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.749512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa70136-7690-4708-8d09-49271757947a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.767263 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-daa70136-7690-4708-8d09-49271757947a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:28.768562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1284/2536] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:28.768686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-daa70136-7690-4708-8d09-49271757947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.769264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.770472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.776319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1253/2537] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:28.776319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa70136-7690-4708-8d09-49271757947a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.779726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-daa70136-7690-4708-8d09-49271757947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.784051 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76d8f3-f5c1-43bb-a1fa-dc0f35f69237 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.788097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.788864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.789654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.789796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.790093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.790883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.792061 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa70136-7690-4708-8d09-49271757947a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.792794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98640aed-516a-4acf-9799-061c9d163291 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.798397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.799085 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.799363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.799636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.800315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.802197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.820785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1285/2538] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:28.822159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.822478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.822767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.831901 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1254/2539] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:28.832852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.841399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:28.842620 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:28.898902 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed40c4-473d-439c-ac04-ea4105ec9aca admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.903835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1286/2540] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:28.904572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.904824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.905482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.905768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.906042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.906731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.908696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.911144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:28.915259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7224d-9b92-47d4-aecd-fa8e434b5c0d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.924198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.924980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.925274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.926083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.926954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.929383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.930395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1255/2541] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:28.934249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.944051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f617dc-42f2-4ad9-8393-cb3452ffe646 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.948825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.949513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.949663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1287/2542] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/users => generated 309 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:28.949849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.950074 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.950335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.950817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:28.951248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.952845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.959642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ef274-3f2b-4474-9c5f-a2111f0f6b96 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.964232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.965447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.965731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.966141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1256/2543] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:28.966255 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.966711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.966928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.968339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.976459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd7f95-88d4-453b-a249-14ad76c2b414 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.979109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1288/2544] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:28.979867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:28.984553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.985196 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.985476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.985752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.986473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:28.988198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:28.991591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267872-ba0d-40fd-8c28-c1ee05451a10 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:28.996882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:28.997501 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:28.997779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:28.998053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:28.998726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.001890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1257/2545] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.001890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.002080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.009040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f231ec-5e1d-4f54-b03f-9eea75194ea0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.014317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.014979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.015262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.015539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.016257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.017816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.030849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1289/2546] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:28 2022] POST /identity/v3/projects => generated 353 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.030849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.031633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1258/2547] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.032566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.043499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af26dc-525b-45f0-bfbc-173922df5bad admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.093554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae038be-112c-4783-8f54-750bcf351910 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.098988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.099872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.100153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.100423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] PATH_INFO: `/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2/users/9808cff312e147448522eb7d872a2928/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.101461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.102114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.102396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.102665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.104071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.105989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.114959 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2d8adbf1694c8e8ad7c42a3838d7a2, user_id=9808cff312e147448522eb7d872a2928, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.117718 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.126731 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1259/2548] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/users => generated 277 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.127545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.130840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1290/2549] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:29 2022] PUT /identity/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2/users/9808cff312e147448522eb7d872a2928/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:29.131690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.140650 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83a22-271e-4808-bbea-b30e7741c6b8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.165179 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.193829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94655-1b1b-4524-84eb-62db24c23e2a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.198506 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.199096 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.199780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.200057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.200322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.200532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.200809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.201038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.201161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] PATH_INFO: `/v3/projects/32a61af2096847dd86e61b85a74a30e2/users/807e2763ae6a41299319bb60096dc77a/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.202627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.213629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1291/2550] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.214738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.215606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a61af2096847dd86e61b85a74a30e2, user_id=807e2763ae6a41299319bb60096dc77a, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.218545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.225298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89672dc8-54ea-466e-8691-86d6a2cd176e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.230288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.231014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.231295 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.232104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.232286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.233249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1260/2551] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:29 2022] PUT /identity/v3/projects/32a61af2096847dd86e61b85a74a30e2/users/807e2763ae6a41299319bb60096dc77a/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:29.234050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.234266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80eca45-030c-4724-af97-db1aec89a9cc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.244679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1292/2552] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.246799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.247265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.247540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.273228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] MFA Rules not processed for user `807e2763ae6a41299319bb60096dc77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:29.274439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:29.283869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.305340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422fabd-272e-458c-ab44-b2ab8e4647db admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.311386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.312043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.312321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.312590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.313312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.315106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.324994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1261/2553] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.325757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.334988 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768f8bd-f28d-49fe-acef-07c984c3f118 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.339599 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.340521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.340803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.341075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] PATH_INFO: `/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2/users/9808cff312e147448522eb7d872a2928/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.341812 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1293/2554] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:29.343291 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2d8adbf1694c8e8ad7c42a3838d7a2, user_id=9808cff312e147448522eb7d872a2928, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.343745 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.346600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.354576 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543ea0-d460-4963-b973-dd9a3c74b585 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.358118 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1262/2555] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:29 2022] PUT /identity/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2/users/9808cff312e147448522eb7d872a2928/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:29.358899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.421897 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.422968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.423254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.423534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] PATH_INFO: `/v3/projects/c5dd0e6014384b79a574778af3f07f57/users/84b0617932d544e39f6cc458a421d238/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.430629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dee25-2b77-4b70-a6dc-4b8963648f40 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.437523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.438207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.438493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.439011 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5dd0e6014384b79a574778af3f07f57, user_id=84b0617932d544e39f6cc458a421d238, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.439130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.439769 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.441673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4888d5-4050-49ff-8de2-044dad2f4f56 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.441782 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcc4e3-227e-4526-83aa-65623c8948cb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.454545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1294/2556] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:29 2022] PUT /identity/v3/projects/c5dd0e6014384b79a574778af3f07f57/users/84b0617932d544e39f6cc458a421d238/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:29.455842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.456127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.456525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.471278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1263/2557] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.472555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.472852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.473153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.485965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] MFA Rules not processed for user `9808cff312e147448522eb7d872a2928`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:29.487214 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:29.501738 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] MFA Rules not processed for user `84b0617932d544e39f6cc458a421d238`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:29.502994 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:29.508019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.561645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1295/2558] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:29.562473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.576264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1264/2559] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:29.577107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.578538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:29.589191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.635030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afa55-04fd-476b-bebf-bf9378d93a2b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.635391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcffd69-ffc2-429c-ae00-718383fde4db None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.639076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.639715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.639998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.640152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.640264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.640978 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.641357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.641728 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.642517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.644787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.649314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.650740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.662613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1265/2560] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:29.663543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.669780 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1296/2561] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.670843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.674938 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eaf7c-4bad-4cba-bbc5-cb487bfc3bf0 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.680756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f328e-205c-4c82-82a2-a9538216f525 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.683531 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.684195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.684483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.684759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.685452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.687068 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.687482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.687652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.688036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.688373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.689382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.692254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.704679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1297/2562] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.708710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.719533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dac607-fe2c-44bf-ae73-02430ff7984a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.724808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.725323 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.725611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.725888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.726613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.728262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.734625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1266/2563] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/projects => generated 355 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.736001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.744765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1298/2564] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.745826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.757996 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a08ce-3c50-4b7e-9b80-7fecee343d9a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.810434 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50945224-577b-4905-935f-6fc11bc5d9d7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.816518 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.817192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.817554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.817763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.818503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.821312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.824224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.825135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.825418 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.825697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] PATH_INFO: `/v3/projects/230a0da654b94d86b70141facffbe2f2/users/726ed422aea74bc685c10c846f354d3c/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.845976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1267/2565] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:29.846209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230a0da654b94d86b70141facffbe2f2, user_id=726ed422aea74bc685c10c846f354d3c, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.846326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.846816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.854397 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7fe74-5a76-461c-81e2-c8c3d3166d38 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.861883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1299/2566] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:29 2022] PUT /identity/v3/projects/230a0da654b94d86b70141facffbe2f2/users/726ed422aea74bc685c10c846f354d3c/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:29.862674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:29.870662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.921406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.922192 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.922475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.922751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] PATH_INFO: `/v3/users/b19ae1793f01402fa3027431791219d6` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.923973 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19ae1793f01402fa3027431791219d6)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.925573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07728-fefc-4a04-8dc5-2c5884c807f4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:29.925905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.930931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:29.931601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.931888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.932163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.932894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:29.934544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc4012-28a7-47e0-a857-bafb7436cb32 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:29.945318 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1300/2567] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:29.947031 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084eb1-748a-42ab-be1a-851c33766ba4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:29.947760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084eb1-748a-42ab-be1a-851c33766ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:29.948634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084eb1-748a-42ab-be1a-851c33766ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:29.962228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:29.974654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19ae1793f01402fa3027431791219d6'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:29.975359 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19ae1793f01402fa3027431791219d6'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:29.975977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19ae1793f01402fa3027431791219d6'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:29.979132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-72084eb1-748a-42ab-be1a-851c33766ba4 None None] MFA Rules not processed for user `726ed422aea74bc685c10c846f354d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:29.980478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-72084eb1-748a-42ab-be1a-851c33766ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:29.984462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19ae1793f01402fa3027431791219d6'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:29.991970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19ae1793f01402fa3027431791219d6'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.010410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1268/2568] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:29 2022] DELETE /identity/v3/users/b19ae1793f01402fa3027431791219d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:30.014200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.015402 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.052886 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1301/2569] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:30.054253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.054622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.054994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.071887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:30.073090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.081348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1302/2570] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:30.082096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.083468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.085230 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faeb334-16a6-4afc-a0ff-3d0fcc33083e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.089571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.091127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.091473 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.091821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] PATH_INFO: `/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e/users/a697a577c4764de289cbc64f4bb5de36/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.096293 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf91ff-b02f-49a7-8eb2-02151b3d7d1f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.101880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.102701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.103023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.103335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.104038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.106085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.107237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f79d9d948ee480c8e4c1353a1e64d0e, user_id=a697a577c4764de289cbc64f4bb5de36, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.109877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.122853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1269/2571] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:30 2022] PUT /identity/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e/users/a697a577c4764de289cbc64f4bb5de36/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:30.123888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.125718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.133187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1303/2572] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:30 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:30.134013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.151369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:30.188635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f9f4d-6bd3-4f8f-a6cf-16ab5339e462 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.197476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e296a-95ae-496a-95ab-73217e89af99 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.203259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.203921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.204205 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.204479 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.205209 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.207180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.221354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1304/2573] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:30.221354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.228726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315f81-9fca-408d-ad03-00a12be102a7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.233808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.234448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.234730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.235168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.235854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.237435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.237757 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.238071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.238347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.238614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.247629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.249056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.259109 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1305/2574] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:30 2022] POST /identity/v3/users => generated 299 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:30.259961 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1270/2575] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:30.263770 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.267134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.273463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c338d7-7e5c-4068-8a52-5ebe66fac98d service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.279351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd04ec-7889-4f3e-8d50-f9c134bfe7c5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.279498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.280130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.280425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.280781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.281455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.283064 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.284377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.285159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.285447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.285731 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] PATH_INFO: `/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e/users/a697a577c4764de289cbc64f4bb5de36/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.287949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f79d9d948ee480c8e4c1353a1e64d0e, user_id=a697a577c4764de289cbc64f4bb5de36, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.290597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d63a5-166a-454b-b7d0-da7aad716f1a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.294509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1271/2576] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:30.298721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.308204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1306/2577] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:30 2022] PUT /identity/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e/users/a697a577c4764de289cbc64f4bb5de36/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:30.317925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1faac0-e3ba-41b6-b4b2-a482695e6fca admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.319542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.319831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.320116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.357986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] MFA Rules not processed for user `a697a577c4764de289cbc64f4bb5de36`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:30.359595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.385208 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.385720 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:30.386509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.387560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.388212 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] PATH_INFO: `/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86/users/454b7a59ba6c403882a583bf1489dc12/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.405934 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c22bb16f6624cbd92f9e1b9e2e24b86, user_id=454b7a59ba6c403882a583bf1489dc12, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.408684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.429273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1272/2578] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:30 2022] PUT /identity/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86/users/454b7a59ba6c403882a583bf1489dc12/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:30.431742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.455424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1307/2579] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:30.467439 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.468815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.501327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:30.515811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8d1a-d631-4511-9101-49b44cc363ca admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.525956 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.526639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.526959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.527268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.528112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.529715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.543707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1273/2580] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:30.544461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bbb9-94dc-4627-9855-1e669728c400 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.544992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.556942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dccbd-32d9-49db-ae40-b825fc4d1c82 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.567879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.568832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.569119 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.569522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] PATH_INFO: `/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86/users/454b7a59ba6c403882a583bf1489dc12/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.572318 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c22bb16f6624cbd92f9e1b9e2e24b86, user_id=454b7a59ba6c403882a583bf1489dc12, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.575690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.591040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1274/2581] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:30 2022] PUT /identity/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86/users/454b7a59ba6c403882a583bf1489dc12/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:30.591040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.592389 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.617441 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.618101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.618394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.618668 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.659763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2b245-7503-4656-9b21-1fe02a91f314 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.665175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.666175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.666567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.666890 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] PATH_INFO: `/v3/projects/dff10b2d0ea3435a8848e73f82c59a1e` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.668446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff10b2d0ea3435a8848e73f82c59a1e)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.670377 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.684707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.686160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e4808-d126-48b4-9189-08c7f866c090 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.696135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1308/2582] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:30.697482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.697785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.698144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.723225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff10b2d0ea3435a8848e73f82c59a1e', 'request_id': 'req-d56ae748-8961-4398-9506-31eade99d38a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.729221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] MFA Rules not processed for user `454b7a59ba6c403882a583bf1489dc12`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:30.730488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.735165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff10b2d0ea3435a8848e73f82c59a1e', 'request_id': 'req-d56ae748-8961-4398-9506-31eade99d38a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.743730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff10b2d0ea3435a8848e73f82c59a1e', 'request_id': 'req-d56ae748-8961-4398-9506-31eade99d38a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.745022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:30.756578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] The token cache is being invalidate because project dff10b2d0ea3435a8848e73f82c59a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:30.757058 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.757620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.759653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1275/2583] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:30 2022] DELETE /identity/v3/projects/dff10b2d0ea3435a8848e73f82c59a1e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:30.763871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.767596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.813387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1309/2584] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:30.813387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.814937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.842766 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae748-8961-4398-9506-31eade99d38a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.851824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.852849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.853147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.853512 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] PATH_INFO: `/v3/users/9808cff312e147448522eb7d872a2928` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.854888 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9808cff312e147448522eb7d872a2928)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.857579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.863070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:30.877162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c3e8c-016f-4996-ab79-8bc46a8dc679 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:30.881182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:30.881835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:30.882547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:30.882547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:30.884265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.908527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9808cff312e147448522eb7d872a2928'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.909203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9808cff312e147448522eb7d872a2928'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.909821 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9808cff312e147448522eb7d872a2928'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.917665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9808cff312e147448522eb7d872a2928'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.925257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9808cff312e147448522eb7d872a2928'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:30.943721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1276/2585] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:30 2022] DELETE /identity/v3/users/9808cff312e147448522eb7d872a2928 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:30.946724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:30.946724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:30.948185 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:30.949919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:30.959942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1310/2586] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:31.011737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3849597-8e31-4f34-bb03-7ec3f0709bc7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.016251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.016950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.017418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.017807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.019722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.092509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.094207 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c955a69-b3d2-4516-88b8-64d202bc0652 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.108541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1277/2587] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:31.264082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.276010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68374b0-7391-4175-95fd-82216caff01e service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.348015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.348888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.349282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.349461 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] PATH_INFO: `/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.350636 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2d8adbf1694c8e8ad7c42a3838d7a2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.352581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.359889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:31.405105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2d8adbf1694c8e8ad7c42a3838d7a2', 'request_id': 'req-82640180-e27f-444f-8333-df9f03b78e8f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:31.410571 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2d8adbf1694c8e8ad7c42a3838d7a2', 'request_id': 'req-82640180-e27f-444f-8333-df9f03b78e8f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:31.420749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2d8adbf1694c8e8ad7c42a3838d7a2', 'request_id': 'req-82640180-e27f-444f-8333-df9f03b78e8f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:31.432488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] The token cache is being invalidate because project 4e2d8adbf1694c8e8ad7c42a3838d7a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:31.432805 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:31.433424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-82640180-e27f-444f-8333-df9f03b78e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:31.435331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1311/2588] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:31 2022] DELETE /identity/v3/projects/4e2d8adbf1694c8e8ad7c42a3838d7a2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:31.439944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.440241 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.440746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.460950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:31.462369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.499395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:31.533997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1278/2589] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:31.541401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.541608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.541919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.560742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:31.562002 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.570869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1312/2590] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:31.575697 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.587388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c8212-9b5c-47b1-a597-85871b9743c4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.593358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.594047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.594335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.594613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.595463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:31.596025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.597688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.609863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1279/2591] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:42:31.614111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.623250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938db88-9cda-4e84-909d-0d5015794a07 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.628335 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.628981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.629264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.629541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.630236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.632161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.654484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.656399 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.659559 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1313/2592] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:31.663972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.715129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27abd2d-ab96-4eb6-835c-748da0e814da admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.727925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924c77-8f5b-45ef-9bbf-7f83c5665b92 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.733238 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.733902 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.734183 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.734486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.735231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.737155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.760971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1314/2593] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:31.762015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.766631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.767210 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.767491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.767767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.769533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.771271 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0d396-2300-402a-a97c-7deaaf6ccb44 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.775776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.776404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.776684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.776957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.777705 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.779308 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.789085 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1315/2594] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:31.793283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.802724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99615-8a35-440c-b6e4-2abb9daf8d6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.808494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.809652 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.809930 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.810226 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.810994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.812604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.823647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1316/2595] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:31.827699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.831282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.832740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab820060-217c-48b4-be6f-a9fb8d9097a5 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.837273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3ef8-5513-411b-b6e7-8b0b12155fd4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:31.841611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1280/2596] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:31.841945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:31.842827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.843154 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.843428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] PATH_INFO: `/v3/projects/e93c5514b27f43f7b6e9c9ccdc8fd74c/users/486b6013fedb4d41bf8cae87cf45f6c9/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.858856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93c5514b27f43f7b6e9c9ccdc8fd74c, user_id=486b6013fedb4d41bf8cae87cf45f6c9, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:31.861489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:31.874974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1317/2597] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:31 2022] PUT /identity/v3/projects/e93c5514b27f43f7b6e9c9ccdc8fd74c/users/486b6013fedb4d41bf8cae87cf45f6c9/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:31.886586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:31.886907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:31.887184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:31.914068 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] MFA Rules not processed for user `486b6013fedb4d41bf8cae87cf45f6c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:31.915292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:31.933922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=100 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:31.983279 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1281/2598] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:31.987649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:31.999257 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:32.058436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa1572-bfd2-4133-9585-97d6fefcf2da admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.063557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.064338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.064476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.064749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.065452 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.067395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.108490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1318/2599] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:31 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:32.109670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.122485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fcc47-ef20-4ecc-8624-c28ba1471323 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.205550 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.206489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.206489 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.206801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.207494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.210056 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.234587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1282/2600] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:32 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:32.235769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.247259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99e0d-806f-4fcd-a861-14519cf13c33 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.257641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.258840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.258840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.259021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.259633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.261287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.281615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1319/2601] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:32.282666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.294715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b386b1-6c57-47c6-8ff8-cdb1ca121590 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.299355 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.300255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.300538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.300809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] PATH_INFO: `/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b/users/6acaa8935b1e45e2907636496a3b7983/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.315612 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb0cbfd7e924b669cd81106acfa7a1b, user_id=6acaa8935b1e45e2907636496a3b7983, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.318286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.332889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1283/2602] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:32 2022] PUT /identity/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b/users/6acaa8935b1e45e2907636496a3b7983/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:32.336898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.347810 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eab41-743e-4695-9d70-834de906459c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.406750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.407453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.407735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.408008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.408881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.410610 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ad95b-764a-4e83-b56d-13ba37e86f16 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.420923 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1320/2603] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:32.425751 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.438962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281d0a-907e-48db-a78d-532ddb0acae8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.445944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.446889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.447171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.447449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] PATH_INFO: `/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b/users/6acaa8935b1e45e2907636496a3b7983/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.449969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb0cbfd7e924b669cd81106acfa7a1b, user_id=6acaa8935b1e45e2907636496a3b7983, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.452657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.463843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1284/2604] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:32 2022] PUT /identity/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b/users/6acaa8935b1e45e2907636496a3b7983/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:32.475642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.476003 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.476348 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.503538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] MFA Rules not processed for user `6acaa8935b1e45e2907636496a3b7983`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:32.504815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:32.575140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1321/2605] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:32.585698 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.587437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:32.645135 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4edc8-6dae-4e9d-a2dd-6207d9468176 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.648736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.649360 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.649635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.649905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.658622 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.660080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.669193 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1285/2606] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:32.765010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:32.770722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:32.775045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5a191-3cc5-4a5b-9c46-faad033a1e70 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:32.779483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:32.780118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:32.780399 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:32.780671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:32.836128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:32.837566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d4781-fa16-497d-a310-552eb22cb581 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:32.846634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1322/2607] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:35.515034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:35.535825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:35.594513 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6491b-4372-48d1-bac4-6995e2150bff service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:35.600130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:35.601152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:35.601538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:35.601923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] PATH_INFO: `/v3/users/486b6013fedb4d41bf8cae87cf45f6c9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:35.603455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] RBAC: Authorizing `identity:delete_user(user_id=486b6013fedb4d41bf8cae87cf45f6c9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:35.606138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:35.662542 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486b6013fedb4d41bf8cae87cf45f6c9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:35.663640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486b6013fedb4d41bf8cae87cf45f6c9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:35.664849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486b6013fedb4d41bf8cae87cf45f6c9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:35.674968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486b6013fedb4d41bf8cae87cf45f6c9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:35.685082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486b6013fedb4d41bf8cae87cf45f6c9'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:35.716312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1286/2608] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:35 2022] DELETE /identity/v3/users/486b6013fedb4d41bf8cae87cf45f6c9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:35.721029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:35.721529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:35.721628 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:35.741038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:35.742263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:35.816518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1323/2609] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:35.826142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:35.827572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:35.891524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa8e4-c18b-40f7-9845-92324e09d9af admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:35.896381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:35.897175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:35.897530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:35.898394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:35.909388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:35.910949 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:35.921764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1287/2610] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:35.982360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:35.983954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:36.047024 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458878e9-c6ad-4bf7-9860-7b4b8b325cbc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:36.051635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:36.052341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:36.052624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:36.052896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:36.054826 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:36.153665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:36.155386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:36.160395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:36.169052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1324/2611] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:36.272946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:36.274824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:36.293625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34d3f1-656d-4b73-8b41-2b5fa3590237 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:36.306432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:36.307558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:36.308238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:36.309809 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] PATH_INFO: `/v3/projects/e93c5514b27f43f7b6e9c9ccdc8fd74c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:36.311686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93c5514b27f43f7b6e9c9ccdc8fd74c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:36.314162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:36.367963 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:36.389156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93c5514b27f43f7b6e9c9ccdc8fd74c', 'request_id': 'req-df0afc6b-bb0b-4c97-82ba-a8e0193641da'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.400437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93c5514b27f43f7b6e9c9ccdc8fd74c', 'request_id': 'req-df0afc6b-bb0b-4c97-82ba-a8e0193641da'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.411332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93c5514b27f43f7b6e9c9ccdc8fd74c', 'request_id': 'req-df0afc6b-bb0b-4c97-82ba-a8e0193641da'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.426948 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] The token cache is being invalidate because project e93c5514b27f43f7b6e9c9ccdc8fd74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:36.427357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.428032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.430530 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1288/2612] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:36 2022] DELETE /identity/v3/projects/e93c5514b27f43f7b6e9c9ccdc8fd74c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:36.435776 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:36.446149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26531183-f5cc-4ae1-b89d-5cea4b393295 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:36.544973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:36.545867 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:36.546508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:36.546508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] PATH_INFO: `/v3/users/6acaa8935b1e45e2907636496a3b7983` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:36.547697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=6acaa8935b1e45e2907636496a3b7983)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:36.550072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:36.646835 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6acaa8935b1e45e2907636496a3b7983'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.648151 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6acaa8935b1e45e2907636496a3b7983'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.689058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6acaa8935b1e45e2907636496a3b7983'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.703034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6acaa8935b1e45e2907636496a3b7983'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.726464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf78a61-a40a-4c8f-898e-ac8aba956cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6acaa8935b1e45e2907636496a3b7983'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:36.739057 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1325/2613] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:36 2022] DELETE /identity/v3/users/6acaa8935b1e45e2907636496a3b7983 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:37.170381 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.171829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:37.185918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0afc6b-bb0b-4c97-82ba-a8e0193641da admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.253867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.254723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.255033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.255305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] PATH_INFO: `/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.256535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb0cbfd7e924b669cd81106acfa7a1b)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.258375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.303030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb0cbfd7e924b669cd81106acfa7a1b', 'request_id': 'req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:37.309289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb0cbfd7e924b669cd81106acfa7a1b', 'request_id': 'req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:37.315989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb0cbfd7e924b669cd81106acfa7a1b', 'request_id': 'req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:37.327776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] The token cache is being invalidate because project 1bb0cbfd7e924b669cd81106acfa7a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:37.328059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:37.328689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9e3cee7c-ea4f-4f59-99cc-e5f0ac64ff46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:37.330247 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1289/2614] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:37 2022] DELETE /identity/v3/projects/1bb0cbfd7e924b669cd81106acfa7a1b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:37.335168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.335458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.335813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.354697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:37.356156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:37.421139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1326/2615] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:37.429222 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.429543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.429823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.450026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:37.451286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:37.460116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1290/2616] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:37.464500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.475006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917760-5de3-46ca-a2bd-3b8a82fcd4a0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.479709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.480364 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.480645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.480918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.481648 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:37.481950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.483589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.497413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1327/2617] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:37.502333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.513521 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5c13-dd07-4585-9c7a-134752ed712d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.518385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.519156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.519470 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.519811 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.520568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.522828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.552337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1291/2618] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:37.553707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.561367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:37.565787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780036a1-daf1-4f62-bada-3d5df443d6e8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.627383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.628049 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.628325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.628595 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.629282 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.631279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.651187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1328/2619] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:37.655334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.665458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6244a-84a2-4599-8398-136c6c4d35f2 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.670744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.671449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.671726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.671996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.672719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.674392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.685378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1292/2620] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:37.689228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.699320 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a8fe-83ac-4aae-a0f2-fa787184afb1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.704100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.704781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.705070 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.705347 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.706110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.707775 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05950b21-3522-46ed-9de8-9186caa24e8c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.719715 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1329/2621] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:37.724080 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.734350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4025c4-cb2c-4c87-986e-83761dd85ca0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.738915 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.739778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.740051 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.740317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] PATH_INFO: `/v3/projects/6fce1efe73c74cb2addd2927c22f25d3/users/962d734302ad453a97e2734babb9b746/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.748778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:37.757985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fce1efe73c74cb2addd2927c22f25d3, user_id=962d734302ad453a97e2734babb9b746, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.760689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.772800 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1293/2622] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:37 2022] PUT /identity/v3/projects/6fce1efe73c74cb2addd2927c22f25d3/users/962d734302ad453a97e2734babb9b746/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:37.783488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.783987 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.784086 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.811676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] MFA Rules not processed for user `962d734302ad453a97e2734babb9b746`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:37.812895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:37.877781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1330/2623] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:37.882186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.944063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddcbe6-8cae-41ab-b427-7b5fa85eab12 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:37.948670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:37.949357 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:37.949630 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:37.949899 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:37.950591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:37.952499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:37.982368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1294/2624] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:37.988876 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:37.995037 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353c44a-4ddc-443c-a770-7db1d020ad25 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.056166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.056838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.057119 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.057396 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.058105 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.060828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.087443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1331/2625] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:37 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:38.088688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.100073 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab685-c82e-4736-86a8-3b23164b1bd1 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.108976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.108976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.108976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.108976 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.109314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.109314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.124219 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1295/2626] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:38.125587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.131388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:38.137368 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e185ad-8639-494c-850b-ccc9edb9c7ec admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.143694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.144859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.145133 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.145407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] PATH_INFO: `/v3/projects/a4b3246b2e5e442aab4222556f141b9b/users/48c219dfe8534ffb99b615e13a2cd433/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.163363 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b3246b2e5e442aab4222556f141b9b, user_id=48c219dfe8534ffb99b615e13a2cd433, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.166319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.183655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1332/2627] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:38 2022] PUT /identity/v3/projects/a4b3246b2e5e442aab4222556f141b9b/users/48c219dfe8534ffb99b615e13a2cd433/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:38.187767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.200802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec26e-69f0-4c5f-84fe-3f0bf6ea4f77 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.319042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.321300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.321300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.321300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.321300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.323024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12030f-6bad-4c22-9304-7b1daa144ebf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.335918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1296/2628] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:38.340144 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.351373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a6b2d-9583-4e50-a5ae-6ac76862a75a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.356829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.357795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.358165 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.358516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] PATH_INFO: `/v3/projects/a4b3246b2e5e442aab4222556f141b9b/users/48c219dfe8534ffb99b615e13a2cd433/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.360635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b3246b2e5e442aab4222556f141b9b, user_id=48c219dfe8534ffb99b615e13a2cd433, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.363423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.376556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1333/2629] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:38 2022] PUT /identity/v3/projects/a4b3246b2e5e442aab4222556f141b9b/users/48c219dfe8534ffb99b615e13a2cd433/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:38.388079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.388373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.388663 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.419802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] MFA Rules not processed for user `48c219dfe8534ffb99b615e13a2cd433`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:38.421154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.475310 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:38.504592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1297/2630] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:38.515774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.518017 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.576375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2793cd-0bb8-4e03-905f-130fa768e149 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.580572 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.581250 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.581575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.581880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.592862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.594306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ca5bc-b738-4e97-94d7-e3edcde33691 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.606617 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1334/2631] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:38.696496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.698047 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.767224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0654c29-fa29-4a92-9340-776f4b382a5c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:38.772062 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:38.772928 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.773206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.773477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] PATH_INFO: `/v3/users/807e2763ae6a41299319bb60096dc77a` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.774677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=807e2763ae6a41299319bb60096dc77a)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:38.776596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:38.830732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807e2763ae6a41299319bb60096dc77a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:38.831410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807e2763ae6a41299319bb60096dc77a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:38.832079 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807e2763ae6a41299319bb60096dc77a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:38.845217 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807e2763ae6a41299319bb60096dc77a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:38.852952 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807e2763ae6a41299319bb60096dc77a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:38.872799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1298/2632] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:38 2022] DELETE /identity/v3/users/807e2763ae6a41299319bb60096dc77a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:38.877593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:38.878055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:38.878152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:38.896424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:38.897615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.967430 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1335/2633] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:38.977538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.979057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.986468 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:38.987849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:38.999989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464560d6-880c-44bd-8cf8-1582269d232b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.004573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.005427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.005703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.005972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] PATH_INFO: `/v3/users/962d734302ad453a97e2734babb9b746` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.007184 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=962d734302ad453a97e2734babb9b746)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.009080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.041606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174dd7d-aa3d-44bb-bc3e-934b9a39da2c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.045808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.046472 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.046750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.047050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.056194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.056827 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962d734302ad453a97e2734babb9b746'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.057491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962d734302ad453a97e2734babb9b746'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.057861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24175feb-177c-45b7-a845-71bafbcba2f2 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.065483 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962d734302ad453a97e2734babb9b746'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.068364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1299/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:39.072817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962d734302ad453a97e2734babb9b746'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.074034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:39.088080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962d734302ad453a97e2734babb9b746'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.094476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1336/2635] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:38 2022] DELETE /identity/v3/users/962d734302ad453a97e2734babb9b746 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:39.099150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.099444 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.099723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.107298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:39.122602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:39.123850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:39.185334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1300/2636] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:39.195375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:39.197101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:39.264842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f0d6-b5e9-41a5-bb26-00c487da94f4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.269729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.270387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.270672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.270992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.281340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.282814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.294256 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1337/2637] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:39.387373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:39.389534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:39.403561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707afe-3af5-4e84-a82e-9e04ea8afb3d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.408845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.409700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.410386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.410699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] PATH_INFO: `/v3/projects/32a61af2096847dd86e61b85a74a30e2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.412214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a61af2096847dd86e61b85a74a30e2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.414639 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.515782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a61af2096847dd86e61b85a74a30e2', 'request_id': 'req-8a542a45-f82f-4d29-a4a4-0738589bc44a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.534964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a61af2096847dd86e61b85a74a30e2', 'request_id': 'req-8a542a45-f82f-4d29-a4a4-0738589bc44a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.545825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a61af2096847dd86e61b85a74a30e2', 'request_id': 'req-8a542a45-f82f-4d29-a4a4-0738589bc44a'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.561620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] The token cache is being invalidate because project 32a61af2096847dd86e61b85a74a30e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:39.561889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.562595 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.570164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1301/2638] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:39 2022] DELETE /identity/v3/projects/32a61af2096847dd86e61b85a74a30e2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:39.570998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:39.582480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d61e63-43a9-43ae-9532-24264e23c7be service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.680561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:39.681945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:39.692983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.693877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.694164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.694606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] PATH_INFO: `/v3/users/726ed422aea74bc685c10c846f354d3c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.696529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=726ed422aea74bc685c10c846f354d3c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.698473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.788410 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726ed422aea74bc685c10c846f354d3c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.789121 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726ed422aea74bc685c10c846f354d3c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.794240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542a45-f82f-4d29-a4a4-0738589bc44a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:39.796606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726ed422aea74bc685c10c846f354d3c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.799831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:39.801141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:39.801459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:39.801732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] PATH_INFO: `/v3/projects/6fce1efe73c74cb2addd2927c22f25d3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:39.802972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fce1efe73c74cb2addd2927c22f25d3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:39.804026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726ed422aea74bc685c10c846f354d3c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.804904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:39.818694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726ed422aea74bc685c10c846f354d3c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.825903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1338/2639] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:39 2022] DELETE /identity/v3/users/726ed422aea74bc685c10c846f354d3c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:39.843373 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:39.859633 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fce1efe73c74cb2addd2927c22f25d3', 'request_id': 'req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.865545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fce1efe73c74cb2addd2927c22f25d3', 'request_id': 'req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.872957 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fce1efe73c74cb2addd2927c22f25d3', 'request_id': 'req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.885300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] The token cache is being invalidate because project 6fce1efe73c74cb2addd2927c22f25d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:39.885684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.886893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:39.888468 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1302/2640] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:39 2022] DELETE /identity/v3/projects/6fce1efe73c74cb2addd2927c22f25d3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:39.892458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:39.894285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:39.902968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:39.975549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ead6b-a764-4454-a86d-87dba97ee361 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:40.033900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:40.034800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:40.035098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:40.035372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] PATH_INFO: `/v3/users/48c219dfe8534ffb99b615e13a2cd433` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:40.036751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c219dfe8534ffb99b615e13a2cd433)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:40.038694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:40.086921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c219dfe8534ffb99b615e13a2cd433'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.087667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c219dfe8534ffb99b615e13a2cd433'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.096911 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c219dfe8534ffb99b615e13a2cd433'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.104557 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c219dfe8534ffb99b615e13a2cd433'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.118575 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c219dfe8534ffb99b615e13a2cd433'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.119216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:40.121492 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:40.126251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1339/2641] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:39 2022] DELETE /identity/v3/users/48c219dfe8534ffb99b615e13a2cd433 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:40.142940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb161d-ce14-4a6c-abca-e175dd4cbfe5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:40.206950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:40.207995 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:40.208496 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:40.208772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] PATH_INFO: `/v3/projects/230a0da654b94d86b70141facffbe2f2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:40.210123 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230a0da654b94d86b70141facffbe2f2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:40.212269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:40.268376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230a0da654b94d86b70141facffbe2f2', 'request_id': 'req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.276059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230a0da654b94d86b70141facffbe2f2', 'request_id': 'req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.288880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230a0da654b94d86b70141facffbe2f2', 'request_id': 'req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.302714 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] The token cache is being invalidate because project 230a0da654b94d86b70141facffbe2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:40.303129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.303870 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.305614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1303/2642] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:40 2022] DELETE /identity/v3/projects/230a0da654b94d86b70141facffbe2f2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:40.310470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:40.312055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:40.326299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c947-45de-4445-b571-e587907f68ec admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:40.355734 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:40.391733 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:40.392633 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:40.392909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:40.393182 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] PATH_INFO: `/v3/users/454b7a59ba6c403882a583bf1489dc12` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:40.394401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=454b7a59ba6c403882a583bf1489dc12)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:40.396375 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:40.446879 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454b7a59ba6c403882a583bf1489dc12'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.447703 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454b7a59ba6c403882a583bf1489dc12'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.455899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454b7a59ba6c403882a583bf1489dc12'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.463774 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454b7a59ba6c403882a583bf1489dc12'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.483781 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0cd73eef-3763-4ee6-aa71-7d31ab6cbf3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454b7a59ba6c403882a583bf1489dc12'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.490852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1340/2643] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:40 2022] DELETE /identity/v3/users/454b7a59ba6c403882a583bf1489dc12 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:40.611180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:40.612783 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:40.625910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca0a7-7913-494b-b0a9-7bfa821604b3 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:40.697155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:40.712694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:40.713569 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:40.713855 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:40.714129 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] PATH_INFO: `/v3/projects/a4b3246b2e5e442aab4222556f141b9b` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:40.715947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b3246b2e5e442aab4222556f141b9b)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:40.717869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:40.770141 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b3246b2e5e442aab4222556f141b9b', 'request_id': 'req-5c402813-8a17-4f09-883e-37a00436ce0f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.777895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b3246b2e5e442aab4222556f141b9b', 'request_id': 'req-5c402813-8a17-4f09-883e-37a00436ce0f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.787546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b3246b2e5e442aab4222556f141b9b', 'request_id': 'req-5c402813-8a17-4f09-883e-37a00436ce0f'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.804557 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] The token cache is being invalidate because project a4b3246b2e5e442aab4222556f141b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:40.804804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.805556 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5c402813-8a17-4f09-883e-37a00436ce0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:40.810665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1304/2644] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:40 2022] DELETE /identity/v3/projects/a4b3246b2e5e442aab4222556f141b9b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:40.813266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:40.813656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:40.814032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:40.840498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:40.844202 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.010146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1341/2645] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:41.019232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.019978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.020436 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.046417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:41.048329 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.060260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1305/2646] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:41.064898 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.078252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c352d-5659-45e7-8b26-99310b8fc8f4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.084707 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.085435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.085862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.086035 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.086880 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:41.087200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.088939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.090593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.092182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.102866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1342/2647] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:41.106888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.119139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e458a-66ee-403e-a1a3-26bf908aed5a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.128234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.129280 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.129551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0c74-5c03-44bd-b27f-fb334fb70b3c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.130149 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.130615 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.131736 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.133932 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.135883 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.137479 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.137799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.138717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] PATH_INFO: `/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.140098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c22bb16f6624cbd92f9e1b9e2e24b86)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.142238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.167383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1343/2648] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:41.171843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.188477 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:41.240031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c22bb16f6624cbd92f9e1b9e2e24b86', 'request_id': 'req-63496a0f-6197-4160-b845-c8aaa47a5869'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:41.255792 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c22bb16f6624cbd92f9e1b9e2e24b86', 'request_id': 'req-63496a0f-6197-4160-b845-c8aaa47a5869'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:41.265687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c02f-f40e-4981-9167-3f0937e36109 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.271274 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c22bb16f6624cbd92f9e1b9e2e24b86', 'request_id': 'req-63496a0f-6197-4160-b845-c8aaa47a5869'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:41.288112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] The token cache is being invalidate because project 2c22bb16f6624cbd92f9e1b9e2e24b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:41.288771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:41.289691 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-63496a0f-6197-4160-b845-c8aaa47a5869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:41.291969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1306/2649] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:41 2022] DELETE /identity/v3/projects/2c22bb16f6624cbd92f9e1b9e2e24b86 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:41.298508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b38d-a910-4fb9-8938-24e34526b299 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.299375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b38d-a910-4fb9-8938-24e34526b299 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.299375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70b38d-a910-4fb9-8938-24e34526b299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.321922 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4e70b38d-a910-4fb9-8938-24e34526b299 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:41.323273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4e70b38d-a910-4fb9-8938-24e34526b299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.376493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.377384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.377786 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.378749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.379711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.382491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.392138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:41.409587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1344/2650] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:41.413624 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.415593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.423013 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1307/2651] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:41.430131 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.430839 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.431863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.432697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3641b97-a541-4c34-921d-4728cd416da6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.439609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.440890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.441551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.442376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.443896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.445676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.457071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1345/2652] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:41.461524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.463554 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:41.465383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.472486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025af8ab-d80b-4e27-b44a-4157fee6fe62 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.475553 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1308/2653] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:41.479653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.481797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.482821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.483112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.483629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.484371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.486232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.491052 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df7ed8-a800-4ebf-b710-a203fb7834a4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.496484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.497591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.497591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.497776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.498412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:41.498723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.500426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.504338 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1346/2654] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:41.508456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.514547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1309/2655] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:41.520055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.522080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7b340-f2c5-4534-a079-dc7e8e6aa0b0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.530054 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.531046 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66ea6a-5cfc-4b9b-8dc6-7efef6d0c1b0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.531219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.531492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.531836 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] PATH_INFO: `/v3/projects/fc84c5caf63f4ecd9d48d8f2e14543af/users/851239fa7e624f858ef7af093173fada/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.537979 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.538717 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.539100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.539451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.540422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.543393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.553191 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc84c5caf63f4ecd9d48d8f2e14543af, user_id=851239fa7e624f858ef7af093173fada, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.555939 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbf062-c2f4-45bf-812f-7d4c8ce96dc2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.571601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1347/2656] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:41 2022] PUT /identity/v3/projects/fc84c5caf63f4ecd9d48d8f2e14543af/users/851239fa7e624f858ef7af093173fada/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:41.578070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1310/2657] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:41.578210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.578210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.578210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.578589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.601634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:41.603007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.650184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a78fa-e1ed-4413-ab88-58e676ca3163 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.657418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.658410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.658410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.658664 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.659398 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.661758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.669765 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1348/2658] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:41.679478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.680994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.687796 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1311/2659] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:41.687967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:41.689248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.699732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90002c7-d39f-4b80-844d-f64193b44629 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.704918 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.705666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.706132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.706515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.707358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.709026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.723651 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1312/2660] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:41.727549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.737787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d90a3-50e2-461b-9d70-49a2b9bdda9b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.742319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.742977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.743260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.743532 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.744160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46fcd3-8886-48ac-b426-8696140ffaf1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.744264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.746395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.748516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.749155 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.749444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.749721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.756462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1313/2661] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:41.760722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:41.763467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.765065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b66a3-8721-40b0-8cef-5427731a762f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.771014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6ee9-aa59-440d-8fc0-e99deac2ce0d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:41.775387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1349/2662] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:41.775643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:41.776498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.776816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.777086 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] PATH_INFO: `/v3/projects/3648da8581f043b8bee2aeb261c8e247/users/f472893818764ffeab2ebf5e4e102483/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.791606 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3648da8581f043b8bee2aeb261c8e247, user_id=f472893818764ffeab2ebf5e4e102483, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:41.794628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:41.808982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1314/2663] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:41 2022] PUT /identity/v3/projects/3648da8581f043b8bee2aeb261c8e247/users/f472893818764ffeab2ebf5e4e102483/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:41.821426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3e7-351f-41ca-b11b-8a4505fd6ffb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:41.821714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3e7-351f-41ca-b11b-8a4505fd6ffb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:41.821991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3e7-351f-41ca-b11b-8a4505fd6ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:41.849849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d19bc3e7-351f-41ca-b11b-8a4505fd6ffb None None] MFA Rules not processed for user `f472893818764ffeab2ebf5e4e102483`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:41.851350 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d19bc3e7-351f-41ca-b11b-8a4505fd6ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:41.926629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1350/2664] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:41.930830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.031069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:42.035112 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b468a0-1b53-47de-b809-6a2dccb367a6 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.041485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.041671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.042098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.042098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.043708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.044653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.044653 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.045654 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.047631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.063480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] MFA Rules not processed for user `2eb34259fb6a4963b9277d2607c21468`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:42.067850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:42.079407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1315/2665] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:41 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:42.083394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.147131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1351/2666] 10.222.0.33 () {58 vars in 967 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:42.150156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530b0e6-a787-40ba-ade4-f8d33d49fb13 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.155831 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.158889 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.159621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.160001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.160959 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.163184 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.170273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.171967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:42.190992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1316/2667] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:42.192138 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.204818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcacbad-569d-45f6-9efe-b58f42bee00c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.211021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.211778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.212124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.212462 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.213272 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.215692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.228451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1317/2668] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:42.231806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:42.232679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.244690 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d79f3-59de-46b7-a242-eec61e49e0b4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.249962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.250726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.251460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.251745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.252490 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.254139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.259770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebd65f-201c-4af9-af3d-1dd7ac3784b4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.270768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1318/2669] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:42.272226 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.283747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717735d-7f6c-4280-a1e0-c1095940ce55 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.289109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.290014 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.290293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.290566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] PATH_INFO: `/v3/projects/d73edc238b964cd382ce5f6a0ee91b39/users/68aa86f20c9b4a4c8256db185891ab60/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.313443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73edc238b964cd382ce5f6a0ee91b39, user_id=68aa86f20c9b4a4c8256db185891ab60, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.316560 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcc349-1cc6-401e-a90f-41d248d28792 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.332502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01bd308fead745c7995c1bcf554cec1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.333166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.333446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.333729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.334841 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1319/2670] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:42 2022] PUT /identity/v3/projects/d73edc238b964cd382ce5f6a0ee91b39/users/68aa86f20c9b4a4c8256db185891ab60/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:42.349605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.349893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.350167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.398372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] MFA Rules not processed for user `68aa86f20c9b4a4c8256db185891ab60`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:42.399817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:42.429473 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.431225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.455060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1352/2671] 10.222.0.33 () {60 vars in 1384 bytes} [Wed Jun 15 00:42:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 00:42:42.517249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1320/2672] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:42.521512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.546464 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8ef1e-bec4-4ff9-8a04-c25b0afaf9b3 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.629649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.630504 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.630695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.631075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.632421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.634337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c7a14-7506-4ba6-b6a3-3972fe3780eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.667823 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1353/2673] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:42.671794 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.691876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615eb96f-ce95-4ad8-911e-72da5aa1691a None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.705992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:42.766508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.767572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.768364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.768364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.774642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.776692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.801988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.802515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.802970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.811681 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1321/2674] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:42.815600 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.830960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c0877-44ac-4437-8c0d-6165d60f744a admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.833102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:42.836728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.837523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.838019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.838395 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.839501 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.841372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.846860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] MFA Rules not processed for user `851239fa7e624f858ef7af093173fada`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:42.848647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:42.859566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1322/2675] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:42.861719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.876715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94ff20-53dc-4ab9-8cc8-a7b97fae2b8d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:42.882642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:42.884065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:42.884493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:42.885144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] PATH_INFO: `/v3/projects/eb82d8f7ae7e42208f67dafeda255974/users/04800326adce4b1492bcbaf81bfc0c69/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:42.912989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb82d8f7ae7e42208f67dafeda255974, user_id=04800326adce4b1492bcbaf81bfc0c69, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:42.916908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:42.940759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1323/2676] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:42 2022] PUT /identity/v3/projects/eb82d8f7ae7e42208f67dafeda255974/users/04800326adce4b1492bcbaf81bfc0c69/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:42.944858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.966423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1354/2677] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:42.969231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:42.983491 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c48c-2c0b-4dd3-b95f-b49faaa2c38b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.028558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6be89-5db1-4edc-8db7-1979ef08d4a5 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.035626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.038189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.038475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.039108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.039876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.041670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.053087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1324/2678] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:43.057197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.060832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.061601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.061957 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.062485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.063436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.065433 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.072287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a03eb-5d18-4c93-b8f3-e292323f8c9c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.077075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.077958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.078234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.078527 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] PATH_INFO: `/v3/projects/eb82d8f7ae7e42208f67dafeda255974/users/04800326adce4b1492bcbaf81bfc0c69/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.080626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb82d8f7ae7e42208f67dafeda255974, user_id=04800326adce4b1492bcbaf81bfc0c69, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.083289 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628deac-7212-4fb7-ac9c-dd9930b6ec13 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.096853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1355/2679] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:42 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:43.097114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1325/2680] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:43 2022] PUT /identity/v3/projects/eb82d8f7ae7e42208f67dafeda255974/users/04800326adce4b1492bcbaf81bfc0c69/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:43.107036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.109950 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.110321 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.110829 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.140858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] MFA Rules not processed for user `04800326adce4b1492bcbaf81bfc0c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:43.142172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:43.178749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce71b-9043-401c-80e1-b930a266660c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.184510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.185421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.185767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.186219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.186949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.188520 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:43.189696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.213453 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1356/2681] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:43 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:43.217941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.223636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1326/2682] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:43.232325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d478e-83b7-478d-9630-94152352c365 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.237459 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.238235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.238612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.239053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.239903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.240022 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.241838 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.241978 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:43.256222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1357/2683] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:43.257234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.264814 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:43.269344 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bbe8d-85a1-43c4-afbd-731c2a6c0279 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.275670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.277498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.278147 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.278536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] PATH_INFO: `/v3/projects/85ce6f7c1ca042038173708d83cf46b9/users/4b9336e2b41a47ecafcdd8733bad2b4e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.299023 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ce6f7c1ca042038173708d83cf46b9, user_id=4b9336e2b41a47ecafcdd8733bad2b4e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.302265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.310326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e01ac2-c78a-4abd-944b-a7b16577d08c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.315148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.315927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.316172 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.316451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.319738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1358/2684] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:43 2022] PUT /identity/v3/projects/85ce6f7c1ca042038173708d83cf46b9/users/4b9336e2b41a47ecafcdd8733bad2b4e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:43.323701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.388240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.389848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.399507 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f82fb-9347-42a4-92de-6d32f765072b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.400330 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1327/2685] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:43.405352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.406022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.406291 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.406561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.407497 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.409036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9815b-6135-4b8b-a7f8-40ebe432e70a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.423644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1359/2686] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:43.427392 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:43.440483 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bead8e4-959c-4ed8-a654-296f9a990dc3 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:43.445806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:43.446693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:43.447007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:43.447396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] PATH_INFO: `/v3/projects/85ce6f7c1ca042038173708d83cf46b9/users/4b9336e2b41a47ecafcdd8733bad2b4e/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:43.449448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ce6f7c1ca042038173708d83cf46b9, user_id=4b9336e2b41a47ecafcdd8733bad2b4e, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:43.452128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:43.463288 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1328/2687] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:43 2022] PUT /identity/v3/projects/85ce6f7c1ca042038173708d83cf46b9/users/4b9336e2b41a47ecafcdd8733bad2b4e/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:44.265025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:44.265555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:44.265555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:44.294510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] MFA Rules not processed for user `4b9336e2b41a47ecafcdd8733bad2b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:44.295834 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:44.384767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1360/2688] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:44.393808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:44.457198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9dc1-33a2-4111-ae12-ca3c46d078f0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:44.460894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:44.461625 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:44.462057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:44.462408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:44.499463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:44.528593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:44.530431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:44.540656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1329/2689] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:45.036156 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:45.037666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:45.051544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aca770-3c0b-47ee-9182-a925ee8d04fc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:45.055327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:45.055984 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:45.056262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:45.056535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:45.140866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:45.142655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:45.150186 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:45.157582 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1361/2690] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:45.806419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:45.815993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b6df-da91-4326-9678-f555b3d1c94e service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:45.820082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:45.820744 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:45.821195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:45.821469 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:45.905781 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:45.907970 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:45.951561 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1330/2691] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:46.023991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:46.035820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0d79-040a-497e-abeb-f73758b98f08 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:46.043979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:46.045758 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:46.046482 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:46.047242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:46.068547 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:46.070537 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d4e-06b8-4de5-9ba7-bfb9ab38ec20 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:46.096721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1362/2692] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:46.871910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:46.873369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:46.954503 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c05db-6ddf-4270-a77e-48a5ead57a0a service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:46.960463 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:46.961536 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:46.961974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:46.962438 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] PATH_INFO: `/v3/users/84b0617932d544e39f6cc458a421d238` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:46.964941 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b0617932d544e39f6cc458a421d238)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:46.967105 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:47.026442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b0617932d544e39f6cc458a421d238'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.028347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b0617932d544e39f6cc458a421d238'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.029405 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b0617932d544e39f6cc458a421d238'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.040621 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b0617932d544e39f6cc458a421d238'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.050125 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b0617932d544e39f6cc458a421d238'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.056165 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:47.075024 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1331/2693] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:46 2022] DELETE /identity/v3/users/84b0617932d544e39f6cc458a421d238 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:47.078730 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:47.080134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:47.080697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:47.100580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:47.101741 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:47.169177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1363/2694] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:47.179640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:47.181429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:47.257482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef6b37-0b5b-4a2b-a827-30b345537bd8 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:47.264106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:47.265233 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:47.265718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:47.266422 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:47.281019 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:47.283012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:47.294753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1332/2695] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:47.715346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:47.719193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:47.737663 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07848feb-951c-42d1-888e-d309ef593ba1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:47.743904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:47.745142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:47.746176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:47.746714 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] PATH_INFO: `/v3/projects/c5dd0e6014384b79a574778af3f07f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:47.748596 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5dd0e6014384b79a574778af3f07f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:47.752264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:47.821251 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5dd0e6014384b79a574778af3f07f57', 'request_id': 'req-b52ba134-91fc-46b5-b58e-fa608109aeb0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.828600 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5dd0e6014384b79a574778af3f07f57', 'request_id': 'req-b52ba134-91fc-46b5-b58e-fa608109aeb0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.836942 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5dd0e6014384b79a574778af3f07f57', 'request_id': 'req-b52ba134-91fc-46b5-b58e-fa608109aeb0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.858089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] The token cache is being invalidate because project c5dd0e6014384b79a574778af3f07f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:47.858297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.859230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:47.863376 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1364/2696] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:47 2022] DELETE /identity/v3/projects/c5dd0e6014384b79a574778af3f07f57 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:47.867797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:47.879566 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e24aa-a3da-4eeb-9f06-297871c4aa52 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:47.959420 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:47.960339 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:47.960588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:47.960850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] PATH_INFO: `/v3/users/a697a577c4764de289cbc64f4bb5de36` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:47.963636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a697a577c4764de289cbc64f4bb5de36)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:47.966570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:48.018572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a697a577c4764de289cbc64f4bb5de36'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.019248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a697a577c4764de289cbc64f4bb5de36'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.019853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a697a577c4764de289cbc64f4bb5de36'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.027755 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a697a577c4764de289cbc64f4bb5de36'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.035803 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c328cc10-c8cc-4b87-be5e-f11da0730c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a697a577c4764de289cbc64f4bb5de36'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.054931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1333/2697] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:47 2022] DELETE /identity/v3/users/a697a577c4764de289cbc64f4bb5de36 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:48.371395 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:48.372927 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:48.407797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ba134-91fc-46b5-b58e-fa608109aeb0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:48.484260 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:48.485990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:48.486598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:48.486937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] PATH_INFO: `/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:48.488868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f79d9d948ee480c8e4c1353a1e64d0e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:48.494696 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:48.507487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:48.567380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f79d9d948ee480c8e4c1353a1e64d0e', 'request_id': 'req-bcf809e2-cd5d-444b-9312-0755a7a45345'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.575401 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f79d9d948ee480c8e4c1353a1e64d0e', 'request_id': 'req-bcf809e2-cd5d-444b-9312-0755a7a45345'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.592370 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f79d9d948ee480c8e4c1353a1e64d0e', 'request_id': 'req-bcf809e2-cd5d-444b-9312-0755a7a45345'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.608082 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] The token cache is being invalidate because project 3f79d9d948ee480c8e4c1353a1e64d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:48.608378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.609326 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-bcf809e2-cd5d-444b-9312-0755a7a45345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:48.612986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1365/2698] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:48 2022] DELETE /identity/v3/projects/3f79d9d948ee480c8e4c1353a1e64d0e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:48.617665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:48.618043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:48.618413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:48.641297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:48.643412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:48.648793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:48.774391 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1334/2699] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:48.782115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:48.782252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:48.782512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:48.805889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:48.807691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:48.826167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:48.833645 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1366/2700] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:48.837830 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a73fd-0366-476a-b3ac-6fc176e800c9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:48.843733 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:48.844684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:48.845065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:48.845771 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:48.846609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:48.847069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:48.848911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:48.867403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1335/2701] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:48.869926 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:48.880278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15325-1e51-462e-b2a0-acd6645f53ae None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:48.885587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:48.887768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:48.888265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:48.888548 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:48.889343 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:48.891470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:48.932756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1367/2702] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:48 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:48.933564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:48.955930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c77cc-0d7e-4bbc-8245-e69996005b26 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.040030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.040997 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.041458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.041734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.042601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.044833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.070563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1336/2703] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:48 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:49.075448 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.091071 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef4ee-0452-4e2b-8cad-556bf79026b2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.097283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.097991 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.098332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.098592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.099328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.101242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.113599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1368/2704] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:49.117423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.129100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad853e-aafc-49d3-9fe7-69fc2d540620 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.135076 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.136936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.137220 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.137498 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.138836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.140461 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc63d0-a3ab-4d22-8d67-2e2af0367291 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.151147 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1337/2705] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:49.157608 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.168918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb527-123d-4878-9421-ffb802564108 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.174938 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.175993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.176563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.177021 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] PATH_INFO: `/v3/projects/81cfa7a4de1648ca9b053e79ec6b85f2/users/d5a5df92be45499e87ae3674a069d0ed/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.196324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cfa7a4de1648ca9b053e79ec6b85f2, user_id=d5a5df92be45499e87ae3674a069d0ed, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.201719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.221044 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1369/2706] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:49 2022] PUT /identity/v3/projects/81cfa7a4de1648ca9b053e79ec6b85f2/users/d5a5df92be45499e87ae3674a069d0ed/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:49.231428 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.231722 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.232001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.274940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] MFA Rules not processed for user `d5a5df92be45499e87ae3674a069d0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:49.279711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:49.368820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1338/2707] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:49.373542 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.448525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36664c3c-4639-4f34-8e5d-c490bd824703 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.457233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.458053 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.458341 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.458660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.459538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.461887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.492398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1370/2708] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:49 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:49.496854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.504042 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:49.509055 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ccb03-7728-4c89-add3-fb03552ba25c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.586998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.587953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.588408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.588776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.590419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.593568 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.621555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1339/2709] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:49 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:49.625612 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.643888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecb19d-94b2-484e-bf7c-2ef204cc9284 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.653278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.654651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.655456 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.656172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.657998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.660498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.675983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1371/2710] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:49.680283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.693088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c72d5-b57e-4621-93dc-8380db039aac admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.698909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.700516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.700947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.701379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] PATH_INFO: `/v3/projects/431c173910334dc3b61f74488850dce2/users/17bf0303205f446096bf4b4263a2c8e5/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.721213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=431c173910334dc3b61f74488850dce2, user_id=17bf0303205f446096bf4b4263a2c8e5, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.725626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.755305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1340/2711] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:49 2022] PUT /identity/v3/projects/431c173910334dc3b61f74488850dce2/users/17bf0303205f446096bf4b4263a2c8e5/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:49.756581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.768169 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beec05-b801-4ec2-a6fa-7909652afcf7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.777535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:49.839300 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.839983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.840454 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.840904 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.841848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.843511 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf63984-9356-4aef-97cb-3d19c12a3d5a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.855139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1372/2712] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:49.859306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:49.872352 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fd6-a38c-4fa6-bf1a-60ab384886dc admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:49.877440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:49.878338 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.878641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.878943 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] PATH_INFO: `/v3/projects/431c173910334dc3b61f74488850dce2/users/17bf0303205f446096bf4b4263a2c8e5/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.880972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=431c173910334dc3b61f74488850dce2, user_id=17bf0303205f446096bf4b4263a2c8e5, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:49.883940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:49.895268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1341/2713] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:49 2022] PUT /identity/v3/projects/431c173910334dc3b61f74488850dce2/users/17bf0303205f446096bf4b4263a2c8e5/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:49.906708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:49.906955 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:49.907244 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:49.940110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] MFA Rules not processed for user `17bf0303205f446096bf4b4263a2c8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:49.941520 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.018231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1373/2714] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:50.030327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.032161 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.114885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91dfea-2096-4985-832f-d66ae60c1581 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.124624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.124624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.124624 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.125523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.137500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.139152 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.151866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1342/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:50.175263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.177471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.202817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.203115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.226029 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f9dbf-0dfa-4aa5-976e-2f8ac2fcf317 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.230693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.231656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.231920 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.232195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.256721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a4b7d-d170-40c8-99bd-2813e2e5bcb7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.262528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.263540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.263882 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.264219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] PATH_INFO: `/v3/users/f472893818764ffeab2ebf5e4e102483` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.265555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f472893818764ffeab2ebf5e4e102483)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.268197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.315594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.317235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.321067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:50.325729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:50.330430 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1343/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:50.331445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.333590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.342872 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f472893818764ffeab2ebf5e4e102483'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.343868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f472893818764ffeab2ebf5e4e102483'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.351334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffecc5-d853-46ae-af34-a8e419958353 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.356407 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f472893818764ffeab2ebf5e4e102483'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.356885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.357907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.358182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.358458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] PATH_INFO: `/v3/users/9dc07d7e2ff644c0942c2aaf5b498656` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.359712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc07d7e2ff644c0942c2aaf5b498656)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.361758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.369762 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f472893818764ffeab2ebf5e4e102483'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.387868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-534a82a6-da67-4cfb-8010-cf1aa8d38bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f472893818764ffeab2ebf5e4e102483'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.400058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1374/2717] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:50 2022] DELETE /identity/v3/users/f472893818764ffeab2ebf5e4e102483 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:50.406999 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.407332 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.407701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.427854 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc07d7e2ff644c0942c2aaf5b498656'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.428785 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc07d7e2ff644c0942c2aaf5b498656'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.429083 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:50.430362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.431331 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc07d7e2ff644c0942c2aaf5b498656'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.447253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc07d7e2ff644c0942c2aaf5b498656'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.460372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc07d7e2ff644c0942c2aaf5b498656'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.500746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1344/2718] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:50 2022] DELETE /identity/v3/users/9dc07d7e2ff644c0942c2aaf5b498656 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:50.520815 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1375/2719] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:50.535077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.537259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.630921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a8af-606f-459e-b858-9c07a468bd77 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.638477 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.640081 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.641385 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.641828 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.749459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.753102 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.782878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1345/2720] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:50.789008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.790549 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:50.811142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15918f-b3ff-404a-8e2f-b498e7027713 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.817444 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:50.818896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:50.818896 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:50.819194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] PATH_INFO: `/v3/projects/10b82c1cb87645d9831a2635a2aa01df` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:50.820701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b82c1cb87645d9831a2635a2aa01df)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:50.823854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:50.903197 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b82c1cb87645d9831a2635a2aa01df', 'request_id': 'req-5bb50005-f124-4c2b-9169-2e0a6c27b867'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.910093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b82c1cb87645d9831a2635a2aa01df', 'request_id': 'req-5bb50005-f124-4c2b-9169-2e0a6c27b867'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.919580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b82c1cb87645d9831a2635a2aa01df', 'request_id': 'req-5bb50005-f124-4c2b-9169-2e0a6c27b867'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.935552 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] The token cache is being invalidate because project 10b82c1cb87645d9831a2635a2aa01df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:50.935910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.936941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:50.939538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1376/2721] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:50 2022] DELETE /identity/v3/projects/10b82c1cb87645d9831a2635a2aa01df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:50.943459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:50.954077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d830a-1652-4872-b8e3-cbeb08d53043 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:50.979618 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:51.028471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.029347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.029629 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.029905 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] PATH_INFO: `/v3/users/21a37f2be5944f8c8d7f4e11b0eff395` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.031133 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a37f2be5944f8c8d7f4e11b0eff395)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.033093 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.080789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a37f2be5944f8c8d7f4e11b0eff395'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.081431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a37f2be5944f8c8d7f4e11b0eff395'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.082015 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a37f2be5944f8c8d7f4e11b0eff395'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.091878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a37f2be5944f8c8d7f4e11b0eff395'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.099974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a37f2be5944f8c8d7f4e11b0eff395'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.115907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.117670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.123703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1346/2722] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:50 2022] DELETE /identity/v3/users/21a37f2be5944f8c8d7f4e11b0eff395 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:51.139424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50005-f124-4c2b-9169-2e0a6c27b867 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.173424 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:51.199895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.200859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.201217 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.201499 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] PATH_INFO: `/v3/projects/3648da8581f043b8bee2aeb261c8e247` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.202726 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3648da8581f043b8bee2aeb261c8e247)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.204664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.256328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3648da8581f043b8bee2aeb261c8e247', 'request_id': 'req-a5833d69-d97f-4360-a2bc-19382505245b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.261850 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3648da8581f043b8bee2aeb261c8e247', 'request_id': 'req-a5833d69-d97f-4360-a2bc-19382505245b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.269708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3648da8581f043b8bee2aeb261c8e247', 'request_id': 'req-a5833d69-d97f-4360-a2bc-19382505245b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.281004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] The token cache is being invalidate because project 3648da8581f043b8bee2aeb261c8e247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:51.281296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.281943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.283767 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1377/2723] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:51 2022] DELETE /identity/v3/projects/3648da8581f043b8bee2aeb261c8e247 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:51.288088 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.289998 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.305765 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bafbe0-c8e6-4e84-a8b3-13d06ed4c3f6 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.315060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.316866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.374144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.375027 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.375341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.375593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] PATH_INFO: `/v3/users/68aa86f20c9b4a4c8256db185891ab60` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.376808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68aa86f20c9b4a4c8256db185891ab60)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.378844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.396030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5833d69-d97f-4360-a2bc-19382505245b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.400713 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.402107 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.402384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.402711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] PATH_INFO: `/v3/projects/ab51d5751c294b7ba2f5faddce141e3e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.404340 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab51d5751c294b7ba2f5faddce141e3e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.406816 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.421442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:51.427615 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68aa86f20c9b4a4c8256db185891ab60'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.428688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68aa86f20c9b4a4c8256db185891ab60'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.429363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68aa86f20c9b4a4c8256db185891ab60'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.438375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68aa86f20c9b4a4c8256db185891ab60'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.445541 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-5e017618-96fc-448c-9c58-a9fbd48f9126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68aa86f20c9b4a4c8256db185891ab60'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.456601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab51d5751c294b7ba2f5faddce141e3e', 'request_id': 'req-b40d45cc-d74b-46eb-a76e-283c121e6841'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.462342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab51d5751c294b7ba2f5faddce141e3e', 'request_id': 'req-b40d45cc-d74b-46eb-a76e-283c121e6841'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.463030 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1347/2724] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:51 2022] DELETE /identity/v3/users/68aa86f20c9b4a4c8256db185891ab60 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:51.474307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab51d5751c294b7ba2f5faddce141e3e', 'request_id': 'req-b40d45cc-d74b-46eb-a76e-283c121e6841'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.476607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:51.488804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] The token cache is being invalidate because project ab51d5751c294b7ba2f5faddce141e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:51.489837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.490986 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b40d45cc-d74b-46eb-a76e-283c121e6841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.496276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1378/2725] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:51 2022] DELETE /identity/v3/projects/ab51d5751c294b7ba2f5faddce141e3e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:51.497981 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.498268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.498548 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.517375 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:51.518602 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.587947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1348/2726] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:51.597413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.601415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.601794 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.622846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:51.624172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.633032 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1379/2727] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:51.638234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.656474 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073371c5-4869-4bea-8081-b1564e78d019 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.661369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.662254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.662693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.663178 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.663900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:51.664269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.666278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.680945 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1349/2728] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:51.686622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.696843 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2200d-b17d-45bb-86b8-ec8d7c0a36eb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.702487 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.703198 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.703621 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.703899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.704599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.706532 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.735974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1380/2729] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:51 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:51.737388 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.748996 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add47bab-e7f9-4e33-b168-36c033a3f705 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.768813 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.770550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:51.820906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.821745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.822121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.822415 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.823198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.825154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.849509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1350/2730] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:51 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:51.853234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.864406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa861e-7e2a-461a-9940-f7b2347e4327 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.864761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b822e-8346-4abb-9947-15ff4d776701 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.869855 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.871452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.871722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.872073 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.872423 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] PATH_INFO: `/v3/projects/d73edc238b964cd382ce5f6a0ee91b39` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.872544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.872804 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.873162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.873823 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.873941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73edc238b964cd382ce5f6a0ee91b39)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.875453 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.875968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.887314 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1351/2731] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:51.890911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.903128 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab2-3007-4a21-b21a-cc00fbd3936f admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.908445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:51.909109 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:51.909390 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:51.909683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:51.910418 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:51.912171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:51.926511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1352/2732] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:51.930656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:51.944593 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc1cd3-9b7d-47a0-9485-8c6017777ec9 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:51.957108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:51.958609 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73edc238b964cd382ce5f6a0ee91b39', 'request_id': 'req-f478ff8c-ddec-445e-a369-b4fedb48c4b5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.972297 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73edc238b964cd382ce5f6a0ee91b39', 'request_id': 'req-f478ff8c-ddec-445e-a369-b4fedb48c4b5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:51.982981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73edc238b964cd382ce5f6a0ee91b39', 'request_id': 'req-f478ff8c-ddec-445e-a369-b4fedb48c4b5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.006842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] The token cache is being invalidate because project d73edc238b964cd382ce5f6a0ee91b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:52.007306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.008112 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.010346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1381/2733] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:51 2022] DELETE /identity/v3/projects/d73edc238b964cd382ce5f6a0ee91b39 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:52.016852 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.019592 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.024694 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.025586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.025875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.026205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] PATH_INFO: `/v3/projects/5f0b03329de24df2b88d07327076ee88/users/7b59f1c6764b48579cc2f3039cd7c08b/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.042619 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0b03329de24df2b88d07327076ee88, user_id=7b59f1c6764b48579cc2f3039cd7c08b, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.043339 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ff8c-ddec-445e-a369-b4fedb48c4b5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.045344 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd3f-ef7b-4c11-a016-0a321ad0a09f admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.055336 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.056385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.056695 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.056943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] PATH_INFO: `/v3/users/04800326adce4b1492bcbaf81bfc0c69` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.061408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1353/2734] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:51 2022] PUT /identity/v3/projects/5f0b03329de24df2b88d07327076ee88/users/7b59f1c6764b48579cc2f3039cd7c08b/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:52.064142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04800326adce4b1492bcbaf81bfc0c69)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.066314 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.067946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.068326 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.068713 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.097059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:52.098269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.138316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04800326adce4b1492bcbaf81bfc0c69'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.139056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04800326adce4b1492bcbaf81bfc0c69'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.143386 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:52.149982 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04800326adce4b1492bcbaf81bfc0c69'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.165420 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04800326adce4b1492bcbaf81bfc0c69'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.186273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04800326adce4b1492bcbaf81bfc0c69'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.189543 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1354/2735] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:52.200316 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.202157 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.207231 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1382/2736] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:52 2022] DELETE /identity/v3/users/04800326adce4b1492bcbaf81bfc0c69 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:52.288611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2cb6-5c91-4b95-aec3-9787e4fdc961 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.380801 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.382001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.382516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.382832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.385322 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.393311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:52.463576 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.465609 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3fc9d-0f62-4350-b957-bee7ef85df40 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.475642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1355/2737] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:52.543489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.545055 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.559230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1990a-6ae3-48da-a959-46596b08d75f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.564768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.565687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.566034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.566403 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] PATH_INFO: `/v3/projects/eb82d8f7ae7e42208f67dafeda255974` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.567692 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb82d8f7ae7e42208f67dafeda255974)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.569613 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.629626 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb82d8f7ae7e42208f67dafeda255974', 'request_id': 'req-04d75c81-e392-4b42-9ccc-45fb3598680b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.636770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb82d8f7ae7e42208f67dafeda255974', 'request_id': 'req-04d75c81-e392-4b42-9ccc-45fb3598680b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.644484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb82d8f7ae7e42208f67dafeda255974', 'request_id': 'req-04d75c81-e392-4b42-9ccc-45fb3598680b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.658221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] The token cache is being invalidate because project eb82d8f7ae7e42208f67dafeda255974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:52.658495 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.659172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-04d75c81-e392-4b42-9ccc-45fb3598680b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:52.660993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1383/2738] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:52 2022] DELETE /identity/v3/projects/eb82d8f7ae7e42208f67dafeda255974 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:52.666044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.666333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.666610 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.684791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:52.686009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.750791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1356/2739] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:52.759221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.759221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.759436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.779152 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:52.780515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:52.789556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1384/2740] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:52.794442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.816962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e04a-1e78-46fb-bfbe-0f90076c2e73 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.822666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.823880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.824167 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.824494 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.825255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:52.825642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.827243 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.844753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1357/2741] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:52.846337 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.857158 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08d495-02ac-4e48-8814-0f7df0cba1fe None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.863465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.864248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.864606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.864952 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.866006 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.868681 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:52.900900 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1385/2742] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:52 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:52.905104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:52.915254 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e12-3235-4743-bb1b-c5aaa4aed30c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:52.975778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:52.976466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:52.976728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:52.977003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:52.977699 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:52.980248 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.005408 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1358/2743] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:52 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:53.006685 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.017204 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca21c-5556-4780-b558-d7ecb4e09056 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.022531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.023232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.023515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.023832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.024518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.026142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.037901 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1386/2744] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:53.042721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.054386 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca769d-da5c-47f8-904e-540d7714fdce admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.059312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.059972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.060256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.060534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.061273 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.062900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613967eb-81e4-41a2-b376-da6d1cc68a1e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.077067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1359/2745] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:53.079041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.096908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c15756-c52f-47ee-8a7e-f1d4d01165ae admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.102536 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.103515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.103797 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.104075 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] PATH_INFO: `/v3/projects/ab28d0c5ac824100bb42a7787276bf25/users/6e1eeb227bf345dfb1d8eec663b9e4c0/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.122135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab28d0c5ac824100bb42a7787276bf25, user_id=6e1eeb227bf345dfb1d8eec663b9e4c0, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.125004 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.141510 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1387/2746] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:53 2022] PUT /identity/v3/projects/ab28d0c5ac824100bb42a7787276bf25/users/6e1eeb227bf345dfb1d8eec663b9e4c0/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:53.152345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.152704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.152980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.174334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:53.186666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] MFA Rules not processed for user `6e1eeb227bf345dfb1d8eec663b9e4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:53.187946 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:53.263203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1360/2747] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:53.267505 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.329751 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:53.352919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dd141-c45d-41c5-85ba-6f0db2d72d98 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.359262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.360173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.360529 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.360920 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.362096 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.364496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.415122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1388/2748] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:53 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:53.418971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.442097 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55068a18-88ef-4fa9-9db0-0e84488ac18c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.580706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.581369 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.581666 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.581989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.582816 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.584761 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.636124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1361/2749] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:53 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:53.641657 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.656604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677f3d7-347f-49ea-aa69-3c066220fea6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.670597 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.672656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.672954 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.673357 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.674228 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.675912 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.688346 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1389/2750] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:53.692960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.706807 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3cb2d-d3d4-46a9-9d8c-a95e8e8749e7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.723522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.723522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.723522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.723522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.724057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.724057 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951b990-15fd-44b3-bc4f-8f94b5ad5dff admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.742685 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1362/2751] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:53.747190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:53.767627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567debf-a7d5-4dde-af95-f2c874409dea admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:53.773821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:53.774950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.775321 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.775820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] PATH_INFO: `/v3/projects/7a78f9d3407f413cb2bd9eae9892b50d/users/4488403677484eeabb52cb3d42e733ff/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.797993 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a78f9d3407f413cb2bd9eae9892b50d, user_id=4488403677484eeabb52cb3d42e733ff, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:53.801060 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:53.832994 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1390/2752] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:53 2022] PUT /identity/v3/projects/7a78f9d3407f413cb2bd9eae9892b50d/users/4488403677484eeabb52cb3d42e733ff/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:53.836435 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:53.836732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:53.837009 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:53.881768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] MFA Rules not processed for user `4488403677484eeabb52cb3d42e733ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:53.883218 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:53.985578 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1363/2753] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:53.990503 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.116039 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e1b1e-f3d4-42d5-85e9-0831101737eb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.122246 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.123097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.123496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.123885 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.124874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.127132 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.164486 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1391/2754] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:53 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:54.169280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.186003 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936757d-a5f0-4e3f-a277-e49dbcae4b49 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.190287 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:54.257585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.260832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.301221 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.301903 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.302186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.302459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.303638 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.305601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.334155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1364/2755] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:54 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:54.338262 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.356607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19ff5-3806-4a2e-9bbc-2e2aab223d43 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.365551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.366240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.366522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.366835 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.367570 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.372002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.387594 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7190a-385d-4222-87da-727690bea362 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.390160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1365/2756] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:54.391535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.396170 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.397208 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.397635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.397989 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] PATH_INFO: `/v3/users/d5a5df92be45499e87ae3674a069d0ed` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.399455 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a5df92be45499e87ae3674a069d0ed)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.402369 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.405818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806c88-83ba-4b36-a822-c0639054b666 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.411647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.412881 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.413443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.413877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] PATH_INFO: `/v3/projects/9103eb11b32a41e484619b89facc4f94/users/82470ebc4fe1465f959ca0cc2aaf6ec4/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.440545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9103eb11b32a41e484619b89facc4f94, user_id=82470ebc4fe1465f959ca0cc2aaf6ec4, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.444857 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.446193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:54.468778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1366/2757] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:54 2022] PUT /identity/v3/projects/9103eb11b32a41e484619b89facc4f94/users/82470ebc4fe1465f959ca0cc2aaf6ec4/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:54.469833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.477200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a5df92be45499e87ae3674a069d0ed'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:54.478022 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a5df92be45499e87ae3674a069d0ed'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:54.487544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a5df92be45499e87ae3674a069d0ed'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:54.501029 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a5df92be45499e87ae3674a069d0ed'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:54.513634 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-39a2d72a-29a0-4374-b050-1492ea33e9ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a5df92be45499e87ae3674a069d0ed'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:54.520222 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1392/2758] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:54 2022] DELETE /identity/v3/users/d5a5df92be45499e87ae3674a069d0ed => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:54.524360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.524744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.525118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.543436 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:54.544654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.549475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df7db-5eeb-4bd7-9c4c-1de8b72b11fa admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.621538 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1393/2759] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:54.625879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.626758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.627063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.627335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.628070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.629657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.633087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.634541 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.642853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1367/2760] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:54.646853 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.648704 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.658715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:54.663034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc9078-4e4c-4d79-9c2f-677da9b0dfbb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.670670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.671832 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.672187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.672535 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] PATH_INFO: `/v3/projects/9103eb11b32a41e484619b89facc4f94/users/82470ebc4fe1465f959ca0cc2aaf6ec4/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.675026 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9103eb11b32a41e484619b89facc4f94, user_id=82470ebc4fe1465f959ca0cc2aaf6ec4, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.677394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c3b77-21e1-487c-bda5-f9055bbac161 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.688627 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1368/2761] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:54 2022] PUT /identity/v3/projects/9103eb11b32a41e484619b89facc4f94/users/82470ebc4fe1465f959ca0cc2aaf6ec4/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:54.698873 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.699166 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.699449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.709569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08ec97-a203-4fde-9d90-315acacd173e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.743265 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] MFA Rules not processed for user `82470ebc4fe1465f959ca0cc2aaf6ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:54.745923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.775079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.777043 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.777653 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.778080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.809851 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:54.844643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1369/2762] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:54.859898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:54.863786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:54.920616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:54.922895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:54.940355 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1394/2763] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:54.973778 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c0b1-5c7f-4632-8ef0-7b85804a5982 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:54.977921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:54.978703 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:54.979124 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:54.979555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:54.981776 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:55.078006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:55.078006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:55.099038 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1370/2764] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:42:55.917845 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:55.920268 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:55.945052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76617013-aafe-4f10-8be5-3d9925e1a04b service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:55.953508 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:55.954544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:55.955102 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:55.956408 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] PATH_INFO: `/v3/projects/81cfa7a4de1648ca9b053e79ec6b85f2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:55.957773 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] RBAC: Authorizing `identity:delete_project(project_id=81cfa7a4de1648ca9b053e79ec6b85f2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:55.959819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:56.020709 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81cfa7a4de1648ca9b053e79ec6b85f2', 'request_id': 'req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.027616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81cfa7a4de1648ca9b053e79ec6b85f2', 'request_id': 'req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.038026 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81cfa7a4de1648ca9b053e79ec6b85f2', 'request_id': 'req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.053727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] The token cache is being invalidate because project 81cfa7a4de1648ca9b053e79ec6b85f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:56.054276 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.054998 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.056601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1395/2765] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:55 2022] DELETE /identity/v3/projects/81cfa7a4de1648ca9b053e79ec6b85f2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:56.060896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:56.072488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44d2c4-8723-46dc-8e78-9c5ef305f3c8 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:56.109442 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:56.160432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:56.161480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:56.161867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:56.162256 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] PATH_INFO: `/v3/users/17bf0303205f446096bf4b4263a2c8e5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:56.163772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bf0303205f446096bf4b4263a2c8e5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:56.166224 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:56.276743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bf0303205f446096bf4b4263a2c8e5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.277944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bf0303205f446096bf4b4263a2c8e5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.279257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bf0303205f446096bf4b4263a2c8e5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.290345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bf0303205f446096bf4b4263a2c8e5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.300989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bf0303205f446096bf4b4263a2c8e5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:56.330111 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1371/2766] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:42:56 2022] DELETE /identity/v3/users/17bf0303205f446096bf4b4263a2c8e5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:56.808432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:56.812598 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:56.884168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b0b6-54a0-49b8-8d04-dbb6da70169c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:56.941710 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:56.949869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:56.951110 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:56.951971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:56.952319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:56.957252 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:57.024181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:57.025825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:57.065181 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.071691 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541396c-9af5-4d39-8bd9-397ada9e6674 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:57.083012 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1396/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:57.142127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136e55-f7fa-40a1-a232-c3b91b90b92d admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:57.153850 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:57.155098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.155586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.156110 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] PATH_INFO: `/v3/projects/431c173910334dc3b61f74488850dce2` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.157659 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=431c173910334dc3b61f74488850dce2)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.160563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:57.234451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431c173910334dc3b61f74488850dce2', 'request_id': 'req-3f013f0b-626d-4369-b954-461b653c1d12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:57.237266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:57.246504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431c173910334dc3b61f74488850dce2', 'request_id': 'req-3f013f0b-626d-4369-b954-461b653c1d12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:57.259871 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431c173910334dc3b61f74488850dce2', 'request_id': 'req-3f013f0b-626d-4369-b954-461b653c1d12'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:57.276083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] The token cache is being invalidate because project 431c173910334dc3b61f74488850dce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:42:57.276844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:57.277573 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3f013f0b-626d-4369-b954-461b653c1d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:42:57.290721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1372/2768] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:42:57 2022] DELETE /identity/v3/projects/431c173910334dc3b61f74488850dce2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:57.298163 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.298533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.298974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.323509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:57.325130 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:57.464821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1397/2769] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:57.473063 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.473363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.473641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.518045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:57.518876 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:57.528417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1373/2770] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:57.533090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:57.546558 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec13fa5-9d2a-4a18-ade6-e2365bc8471b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:57.560873 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:57.561809 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.563172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.563819 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.564683 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:42:57.565097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.567108 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:57.608992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1398/2771] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:42:57.613581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:57.625927 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113318c-89f5-4dcd-8425-e75f56d1220f None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:57.636707 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:57.637587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.637863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.638686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.639433 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.642936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86638fd-a73f-43a9-b29d-e2eb0d4c52d1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:57.709491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1374/2772] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:57 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:57.709742 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:57.721345 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dd15b-002e-4a7a-8eec-621af887620c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:57.884180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:57.884892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.885175 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.885642 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.887378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.894494 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:57.901021 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.903006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.903443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.935233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1399/2773] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:57 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:57.938140 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:57.959544 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] MFA Rules not processed for user `7b59f1c6764b48579cc2f3039cd7c08b`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:57.960038 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427235-4119-4c4b-903f-6ecc02d992f6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:57.961414 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:57.976639 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:57.977330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:57.977618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:57.977888 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:57.980392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:57.982384 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.012115 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1400/2774] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:58.013869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.038968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2561-b94e-40ac-97b4-a3d8f79025a3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.055438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.056135 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.056426 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.056697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.058194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.061334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.064462 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:58.094216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1401/2775] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:42:58.095976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.097859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:58.103367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1375/2776] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:42:58.104740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.126028 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb911a7-4a31-4d50-9dad-74db72dc113d None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.142563 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.143906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.144194 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.144488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] PATH_INFO: `/v3/projects/9cf2f6acd7d34232bbc76b97da0adf09/users/bcac5d12c39443478cff94b71f2b30f4/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.165716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80146-fa00-423e-91d6-0f3640dfbece admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.171550 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.172241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.172701 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.173288 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.175898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf2f6acd7d34232bbc76b97da0adf09, user_id=bcac5d12c39443478cff94b71f2b30f4, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.177562 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:58.183741 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.218680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1376/2777] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:58 2022] PUT /identity/v3/projects/9cf2f6acd7d34232bbc76b97da0adf09/users/bcac5d12c39443478cff94b71f2b30f4/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:58.219867 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.222292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:58.293153 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.294632 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e067ccd-09db-4ac3-9d67-0bfed238e5d2 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.313224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1402/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:42:58.317857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.318161 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.318443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.359450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214eb4c-6944-4345-932e-82826e412cd0 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.361684 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] MFA Rules not processed for user `bcac5d12c39443478cff94b71f2b30f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:58.363402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:58.369082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.369817 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.370107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.370387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.371257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.380286 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.424897 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1377/2779] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:58 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:58.426607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.433002 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:58.504383 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1403/2780] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:58.508842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.525199 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7908241-4ff3-4f9b-99bd-585ff09c2823 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.534969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b1564-1759-4387-8b8a-01bcb69c2e00 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.535580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.536259 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.536545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.536883 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.537525 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.538419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.539116 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.539396 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.539493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.539668 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.540353 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.543793 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.565757 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1378/2781] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:58 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:58.569893 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.574220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1404/2782] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:42:58 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:42:58.575312 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.584033 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8c3fa-94f2-4ca1-b293-3bafc460c71e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.603009 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:58.661331 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde7ce8-2083-409f-b73b-a23311f738e2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.661510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.663670 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.664023 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.664379 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.665205 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.667441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.667779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.668465 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.668749 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.669030 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.669754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.671759 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.680139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1379/2783] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:58.685605 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.692854 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1405/2784] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:42:58 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:42:58.696735 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.699380 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18c96-f44a-428b-a1b3-54a55d155d07 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.704820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.705728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.706011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.706337 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] PATH_INFO: `/v3/projects/717858089df841db9420c6ce17671580/users/e1362472741448d5b93dc1c4b816fc39/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.708381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7055788-455e-40e5-8138-2d24354fa955 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.713127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.714359 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.714641 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.714946 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.715676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.717261 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.723082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=717858089df841db9420c6ce17671580, user_id=e1362472741448d5b93dc1c4b816fc39, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.725768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.727903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1406/2785] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:58.731971 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.741944 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1380/2786] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:58 2022] PUT /identity/v3/projects/717858089df841db9420c6ce17671580/users/e1362472741448d5b93dc1c4b816fc39/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:42:58.742658 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a5d3-3965-4e27-9699-be4deb599367 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.746572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.805869 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.806763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.807077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.807381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] PATH_INFO: `/v3/projects/163cb2e6014841418d68ad423b520e40/users/89933b83d33c460981174206d0215ef3/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.815669 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856663b0-f826-4f8c-85fc-ab405d941fcf admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.821481 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.822155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.822452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.822712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.823858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163cb2e6014841418d68ad423b520e40, user_id=89933b83d33c460981174206d0215ef3, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.824036 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.825611 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.826522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.831077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:58.837806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1381/2787] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:58.842545 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.843122 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1407/2788] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:58 2022] PUT /identity/v3/projects/163cb2e6014841418d68ad423b520e40/users/89933b83d33c460981174206d0215ef3/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:58.847616 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.854177 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6e94-c9fa-40f8-8572-6bd8ebfc81fd admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.921189 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e85d8-cd96-445e-85d3-8343be776167 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.922587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.923729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.924017 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.924296 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] PATH_INFO: `/v3/projects/717858089df841db9420c6ce17671580/users/e1362472741448d5b93dc1c4b816fc39/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.926298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717858089df841db9420c6ce17671580, user_id=e1362472741448d5b93dc1c4b816fc39, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.926674 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:58.927564 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:58.927848 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:58.928124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:58.928874 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:58.929067 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2a853-5618-44aa-a254-760c00de4630 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.930607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:58.942207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1408/2789] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 00:42:58.946042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:58.947049 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1382/2790] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:58 2022] PUT /identity/v3/projects/717858089df841db9420c6ce17671580/users/e1362472741448d5b93dc1c4b816fc39/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:58.958488 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623186c3-7d9b-4581-a3bc-dc46f8cb3e24 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:58.971111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:42:59.022093 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:59.023019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:59.023328 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:59.023588 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] PATH_INFO: `/v3/projects/163cb2e6014841418d68ad423b520e40/users/89933b83d33c460981174206d0215ef3/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:59.025988 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163cb2e6014841418d68ad423b520e40, user_id=89933b83d33c460981174206d0215ef3, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:59.029382 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:59.041048 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1409/2791] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:42:58 2022] PUT /identity/v3/projects/163cb2e6014841418d68ad423b520e40/users/89933b83d33c460981174206d0215ef3/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:42:59.051730 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:59.051900 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:59.052092 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:59.087837 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] MFA Rules not processed for user `89933b83d33c460981174206d0215ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:42:59.089426 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:42:59.173244 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1383/2792] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:42:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:42:59.185377 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:42:59.249114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fff8b5-f43f-44fc-a006-a6afd4444d03 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:42:59.254193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:42:59.254890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:42:59.255173 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:42:59.255447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:42:59.329593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:42:59.332098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:42:59.344042 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1410/2793] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:42:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:02.342311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:02.451856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:02.453380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:02.499443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b7f9-5ac0-4de7-b863-7334a8a49569 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:02.505448 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:02.508607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:02.509078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:02.509547 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] PATH_INFO: `/v3/users/bcac5d12c39443478cff94b71f2b30f4` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:02.512300 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcac5d12c39443478cff94b71f2b30f4)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:02.515285 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:02.580864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:02.618515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a45f-21b4-41ca-a934-c6e3fcb1b64f service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:02.624880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcac5d12c39443478cff94b71f2b30f4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.626412 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcac5d12c39443478cff94b71f2b30f4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.626565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:02.627203 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcac5d12c39443478cff94b71f2b30f4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.628429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:02.628977 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:02.629649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] PATH_INFO: `/v3/users/851239fa7e624f858ef7af093173fada` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:02.632241 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] RBAC: Authorizing `identity:delete_user(user_id=851239fa7e624f858ef7af093173fada)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:02.634970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:02.642050 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcac5d12c39443478cff94b71f2b30f4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.647235 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:02.664540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-c07ee6c1-5608-4c6d-ba17-b21d2f52d925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcac5d12c39443478cff94b71f2b30f4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.718720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1384/2794] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:02 2022] DELETE /identity/v3/users/bcac5d12c39443478cff94b71f2b30f4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:02.726865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:02.728130 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:02.728636 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:02.755984 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:02.757896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:02.792404 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851239fa7e624f858ef7af093173fada'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.794822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851239fa7e624f858ef7af093173fada'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.809064 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851239fa7e624f858ef7af093173fada'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.828249 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851239fa7e624f858ef7af093173fada'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.861221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851239fa7e624f858ef7af093173fada'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:02.878005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1411/2795] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:02 2022] DELETE /identity/v3/users/851239fa7e624f858ef7af093173fada => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:02.878005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:02.881577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:02.899404 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1385/2796] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:02.924594 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:02.926806 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:02.996693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bbc55-c2fc-4a44-9f0c-c7265547441a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.010084 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.011544 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.011831 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.012111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] PATH_INFO: `/v3/users/6e1eeb227bf345dfb1d8eec663b9e4c0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.015599 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1eeb227bf345dfb1d8eec663b9e4c0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.017877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.073511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc92f86-f7cf-4e8f-ab21-144c4df4a7b9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.114172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1eeb227bf345dfb1d8eec663b9e4c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.116732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1eeb227bf345dfb1d8eec663b9e4c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.132365 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1eeb227bf345dfb1d8eec663b9e4c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.146514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1eeb227bf345dfb1d8eec663b9e4c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.176273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1c1c6d72-16f7-47c4-9093-7faf8847334e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1eeb227bf345dfb1d8eec663b9e4c0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.179381 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:03.185964 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.188154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.188154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.188581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.188837 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1412/2797] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:02 2022] DELETE /identity/v3/users/6e1eeb227bf345dfb1d8eec663b9e4c0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:03.191004 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:03.191915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.192489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.193014 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.216445 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.222316 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:03.226692 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.229723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:03.240364 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:03.260417 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1386/2798] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:03.263596 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:03.267065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:03.296592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f86df3-692a-41e1-96a0-43f6a55b59e0 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.367114 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1413/2799] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:43:03.381992 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:03.383779 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:03.451980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.453407 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.455158 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.455406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] PATH_INFO: `/v3/projects/fc84c5caf63f4ecd9d48d8f2e14543af` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.459104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc84c5caf63f4ecd9d48d8f2e14543af)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.462983 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.474589 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1845865-c4d5-45d1-9ac3-7994fddf751d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.481117 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.481821 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.482069 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.482351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.493275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.495393 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.509279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1414/2800] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:03.593239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc84c5caf63f4ecd9d48d8f2e14543af', 'request_id': 'req-747484e8-f82c-47b5-bcc2-bb4330732157'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.603990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc84c5caf63f4ecd9d48d8f2e14543af', 'request_id': 'req-747484e8-f82c-47b5-bcc2-bb4330732157'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.616451 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc84c5caf63f4ecd9d48d8f2e14543af', 'request_id': 'req-747484e8-f82c-47b5-bcc2-bb4330732157'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.636108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] The token cache is being invalidate because project fc84c5caf63f4ecd9d48d8f2e14543af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:03.636409 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.637083 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.639394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1387/2801] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:03 2022] DELETE /identity/v3/projects/fc84c5caf63f4ecd9d48d8f2e14543af => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:03.643406 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:03.661517 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c85e7-e2fb-4d52-a527-813ca84cc872 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.741097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.742513 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.743168 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.743887 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] PATH_INFO: `/v3/users/4b9336e2b41a47ecafcdd8733bad2b4e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.745948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9336e2b41a47ecafcdd8733bad2b4e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.748881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.851104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:03.852540 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:03.869587 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747484e8-f82c-47b5-bcc2-bb4330732157 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:03.882392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:03.882993 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:03.885696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:03.886237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:03.886237 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] PATH_INFO: `/v3/projects/9cf2f6acd7d34232bbc76b97da0adf09` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:03.889403 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf2f6acd7d34232bbc76b97da0adf09)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:03.890555 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9336e2b41a47ecafcdd8733bad2b4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.891754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9336e2b41a47ecafcdd8733bad2b4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.894977 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:03.908807 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9336e2b41a47ecafcdd8733bad2b4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.918207 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9336e2b41a47ecafcdd8733bad2b4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.949309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9336e2b41a47ecafcdd8733bad2b4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:03.957916 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1415/2802] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:03 2022] DELETE /identity/v3/users/4b9336e2b41a47ecafcdd8733bad2b4e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:43:03.992591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf2f6acd7d34232bbc76b97da0adf09', 'request_id': 'req-3252a163-7929-4288-b1cd-556541195d15'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.001146 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf2f6acd7d34232bbc76b97da0adf09', 'request_id': 'req-3252a163-7929-4288-b1cd-556541195d15'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.014930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf2f6acd7d34232bbc76b97da0adf09', 'request_id': 'req-3252a163-7929-4288-b1cd-556541195d15'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.038149 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] The token cache is being invalidate because project 9cf2f6acd7d34232bbc76b97da0adf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:04.038522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.039215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.044511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1388/2803] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:03 2022] DELETE /identity/v3/projects/9cf2f6acd7d34232bbc76b97da0adf09 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:04.047966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.049415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.165990 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae77c-3250-45f2-aae6-9afbf25e4ddd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.194620 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.196843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.230723 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:04.306627 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.308296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.308580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.308856 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] PATH_INFO: `/v3/users/89933b83d33c460981174206d0215ef3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.310313 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89933b83d33c460981174206d0215ef3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.311719 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a163-7929-4288-b1cd-556541195d15 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.313118 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.317106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.317972 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.318261 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.318628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] PATH_INFO: `/v3/projects/85ce6f7c1ca042038173708d83cf46b9` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.319868 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ce6f7c1ca042038173708d83cf46b9)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.322041 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.386306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ce6f7c1ca042038173708d83cf46b9', 'request_id': 'req-e906bb6d-0f4c-4623-a177-d922bdc840a2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.391833 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89933b83d33c460981174206d0215ef3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.393019 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89933b83d33c460981174206d0215ef3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.396456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ce6f7c1ca042038173708d83cf46b9', 'request_id': 'req-e906bb6d-0f4c-4623-a177-d922bdc840a2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.407999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ce6f7c1ca042038173708d83cf46b9', 'request_id': 'req-e906bb6d-0f4c-4623-a177-d922bdc840a2'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.410671 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89933b83d33c460981174206d0215ef3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.420585 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89933b83d33c460981174206d0215ef3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.422327 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] The token cache is being invalidate because project 85ce6f7c1ca042038173708d83cf46b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:04.422700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.423777 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e906bb6d-0f4c-4623-a177-d922bdc840a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.429992 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1389/2804] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:04 2022] DELETE /identity/v3/projects/85ce6f7c1ca042038173708d83cf46b9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:04.431914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe448-3054-4f24-b995-1005e2f987dd None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.432332 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe448-3054-4f24-b995-1005e2f987dd None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.432687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe448-3054-4f24-b995-1005e2f987dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.433531 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:04.444606 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89933b83d33c460981174206d0215ef3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.474304 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1416/2805] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:04 2022] DELETE /identity/v3/users/89933b83d33c460981174206d0215ef3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:04.475522 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.477631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.479604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-089fe448-3054-4f24-b995-1005e2f987dd None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:04.482260 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-089fe448-3054-4f24-b995-1005e2f987dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.567581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ee740-2754-4c22-ac68-77c72dd66b85 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.574233 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.575356 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.575688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.576230 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] PATH_INFO: `/v3/projects/ab28d0c5ac824100bb42a7787276bf25` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.577909 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab28d0c5ac824100bb42a7787276bf25)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.580480 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.603486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1390/2806] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:04.605416 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.605712 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.605989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.636597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:04.637907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.661470 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab28d0c5ac824100bb42a7787276bf25', 'request_id': 'req-775b251d-ffe4-49cc-9038-91c83e654c5d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.670509 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab28d0c5ac824100bb42a7787276bf25', 'request_id': 'req-775b251d-ffe4-49cc-9038-91c83e654c5d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.681177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab28d0c5ac824100bb42a7787276bf25', 'request_id': 'req-775b251d-ffe4-49cc-9038-91c83e654c5d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.697492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] The token cache is being invalidate because project ab28d0c5ac824100bb42a7787276bf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:04.698489 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.699439 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:04.699591 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.701729 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1417/2807] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:04 2022] DELETE /identity/v3/projects/ab28d0c5ac824100bb42a7787276bf25 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:04.709419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.715753 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:04.725040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1391/2808] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:04.729431 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.739588 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154fb33-93fe-4d65-ad1a-461735c42dee None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.740690 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b251d-ffe4-49cc-9038-91c83e654c5d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.744947 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.745745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.746150 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.746526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.747419 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:43:04.747824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.748974 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.750020 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.750360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.751005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.751435 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] PATH_INFO: `/v3/users/4488403677484eeabb52cb3d42e733ff` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.753711 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4488403677484eeabb52cb3d42e733ff)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.768564 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1392/2809] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 00:43:04.769171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.773107 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.785677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543af3a1-88f8-443b-ab62-766599cfc9e1 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:04.798214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:04.799089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:04.799471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:04.799820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:04.800628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:04.807372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:04.864545 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4488403677484eeabb52cb3d42e733ff'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.865374 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4488403677484eeabb52cb3d42e733ff'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.877772 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1393/2810] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:43:04 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:04.879232 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:04.884910 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4488403677484eeabb52cb3d42e733ff'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.915754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4488403677484eeabb52cb3d42e733ff'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.938090 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:04.961981 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4488403677484eeabb52cb3d42e733ff'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:04.977164 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1418/2811] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:04 2022] DELETE /identity/v3/users/4488403677484eeabb52cb3d42e733ff => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:05.040782 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ccd8c-dd7a-46c1-b627-fd00574464ad admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.152608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.154456 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.154750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.155069 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.156251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.160471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.174849 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.182829 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.233101 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46efeec-dd3a-407a-8149-196ca4aa6248 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.236607 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1394/2812] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:43:04 2022] POST /identity/v3/users => generated 287 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:43:05.243007 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.248201 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.250050 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.252354 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.252647 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.253437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] PATH_INFO: `/v3/projects/163cb2e6014841418d68ad423b520e40` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.263655 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163cb2e6014841418d68ad423b520e40)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.273111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.289689 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e61f8-8db7-47cd-a55d-6751b7109b50 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.301917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.303115 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.303401 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.303684 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.304736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.306374 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.349478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1395/2813] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:05.353747 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.358077 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.388572 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4298-ead7-4de4-91b3-876d9e70e24e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.399540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163cb2e6014841418d68ad423b520e40', 'request_id': 'req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.412402 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163cb2e6014841418d68ad423b520e40', 'request_id': 'req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.429694 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163cb2e6014841418d68ad423b520e40', 'request_id': 'req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.435640 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:05.447421 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] The token cache is being invalidate because project 163cb2e6014841418d68ad423b520e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:05.448028 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.448846 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.451367 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1419/2814] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:05 2022] DELETE /identity/v3/projects/163cb2e6014841418d68ad423b520e40 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:05.452390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.453949 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.506235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.507480 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.507614 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.507815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] PATH_INFO: `/v3/projects/7a78f9d3407f413cb2bd9eae9892b50d` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.512249 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a78f9d3407f413cb2bd9eae9892b50d)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.517910 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.597601 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60dd65-f6a3-4c6b-8de2-2b1fbf280c24 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.609554 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.610661 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.611126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.611580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.619580 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.625327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.643061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1420/2815] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:05.650492 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.664889 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804cc6-e415-4fba-8fe6-77fe85b06dc6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.679754 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a78f9d3407f413cb2bd9eae9892b50d', 'request_id': 'req-9ed7774f-28df-4790-b4f4-973d36dcfcf7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.700917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a78f9d3407f413cb2bd9eae9892b50d', 'request_id': 'req-9ed7774f-28df-4790-b4f4-973d36dcfcf7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.726259 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a78f9d3407f413cb2bd9eae9892b50d', 'request_id': 'req-9ed7774f-28df-4790-b4f4-973d36dcfcf7'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.759515 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] The token cache is being invalidate because project 7a78f9d3407f413cb2bd9eae9892b50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:05.760894 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.761136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:05.767951 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1396/2816] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:05 2022] DELETE /identity/v3/projects/7a78f9d3407f413cb2bd9eae9892b50d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:05.772726 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:05.775851 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.823754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.824947 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.825500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.825980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] PATH_INFO: `/v3/projects/a057a550c0fb442ebaa9deb5ced441ba/users/1a0ae44751124e81b32abee9af7494c5/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.848429 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a057a550c0fb442ebaa9deb5ced441ba, user_id=1a0ae44751124e81b32abee9af7494c5, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.851574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b057-6789-493e-b13a-7ceba3d79431 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:05.855565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:05.887651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1421/2817] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:05 2022] PUT /identity/v3/projects/a057a550c0fb442ebaa9deb5ced441ba/users/1a0ae44751124e81b32abee9af7494c5/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:05.890312 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7774f-28df-4790-b4f4-973d36dcfcf7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:05.899287 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d4e5a-3635-435d-ad1d-4cea0a01c30f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.903134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d4e5a-3635-435d-ad1d-4cea0a01c30f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.903467 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d4e5a-3635-435d-ad1d-4cea0a01c30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.969471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-783d4e5a-3635-435d-ad1d-4cea0a01c30f None None] MFA Rules not processed for user `1a0ae44751124e81b32abee9af7494c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:05.971142 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-783d4e5a-3635-435d-ad1d-4cea0a01c30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:05.987533 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:05.988484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:05.988759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:05.989089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] PATH_INFO: `/v3/users/82470ebc4fe1465f959ca0cc2aaf6ec4` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:05.990333 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=82470ebc4fe1465f959ca0cc2aaf6ec4)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:05.992315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:06.007016 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:06.062895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82470ebc4fe1465f959ca0cc2aaf6ec4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.063709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82470ebc4fe1465f959ca0cc2aaf6ec4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.064384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82470ebc4fe1465f959ca0cc2aaf6ec4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.069842 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1422/2818] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:06.071688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.071970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.072593 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.075045 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82470ebc4fe1465f959ca0cc2aaf6ec4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.087586 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82470ebc4fe1465f959ca0cc2aaf6ec4'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.102013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] MFA Rules not processed for user `e1362472741448d5b93dc1c4b816fc39`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:06.103210 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.110815 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1397/2819] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:05 2022] DELETE /identity/v3/users/82470ebc4fe1465f959ca0cc2aaf6ec4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:43:06.123768 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:06.128447 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.215412 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1423/2820] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:06.220163 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d999-53c3-4adf-88b7-97e86748c26c admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:06.233533 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:06.235239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.239960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:06.241516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.241516 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.241746 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.244198 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.359202 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:06.360936 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:06.368225 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adf3bb-02d9-4b06-924b-05803154ed39 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:06.372866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:06.375940 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1398/2821] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:06.453342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:06.453983 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.454265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.454540 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.456553 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.534700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:06.536432 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.576631 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:06.580679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253bd3e-8658-467e-bcd0-8b7fab43b3f8 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:06.602005 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1424/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:06.610987 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:06.640117 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96da98-cacc-405f-9e0e-4b7aa02f40df service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:06.644971 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:06.645836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.646345 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.647053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] PATH_INFO: `/v3/projects/9103eb11b32a41e484619b89facc4f94` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.648012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9103eb11b32a41e484619b89facc4f94)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:06.650255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:06.742732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9103eb11b32a41e484619b89facc4f94', 'request_id': 'req-8a976cfe-d151-41ff-907a-877c06001d5e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.754863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9103eb11b32a41e484619b89facc4f94', 'request_id': 'req-8a976cfe-d151-41ff-907a-877c06001d5e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.769034 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9103eb11b32a41e484619b89facc4f94', 'request_id': 'req-8a976cfe-d151-41ff-907a-877c06001d5e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.784306 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] The token cache is being invalidate because project 9103eb11b32a41e484619b89facc4f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:06.784597 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.785269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-8a976cfe-d151-41ff-907a-877c06001d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:06.786866 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1399/2823] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:06 2022] DELETE /identity/v3/projects/9103eb11b32a41e484619b89facc4f94 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:43:06.792388 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.792688 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.792970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.815656 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:06.818362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.916785 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1425/2824] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:06.926142 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.926457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.926736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.947341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:06.949641 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:06.958732 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1400/2825] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:06.963239 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:06.981787 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af1ec8-9828-483b-b1c1-57904ddf338a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:06.989146 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:06.990270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:06.990716 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:06.991162 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:06.992066 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:43:06.992747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:06.995556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.014240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1426/2826] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:43:07.018848 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.034802 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91cb1e-7bc7-45a3-97aa-496b61c516c9 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.040251 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.040921 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.041197 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.041493 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.042255 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.044637 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.073912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1401/2827] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:43:07 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:07.078242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.090008 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34dc6-4fae-4222-95dc-119a8f6a46dc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.134472 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:07.174432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.175820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.176111 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.176394 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.177682 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.179721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.204744 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1427/2828] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:43:07 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:07.208485 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.219370 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a64e5-6147-4cbe-bfa2-734d88dfb5ce admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.229458 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.230266 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.230656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.231078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.231965 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.233725 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.250443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1402/2829] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:07.251892 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.263667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32ac62-d21b-4be0-b26a-0a57c8472750 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.271286 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.272036 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.272385 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.272722 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.273804 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.275853 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68903f-fb39-467f-9852-ceb46b148b56 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.291082 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.291972 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1428/2830] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:07.302762 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65261e8e-e241-4912-b6b4-cbdae12e95f7 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.309140 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.310591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.310591 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.310861 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] PATH_INFO: `/v3/projects/76d247588aec429b89311037211e81d0/users/01b04eb348374397b58ec5cb16d64171/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.328476 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d247588aec429b89311037211e81d0, user_id=01b04eb348374397b58ec5cb16d64171, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.331701 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.362805 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1403/2831] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:07 2022] PUT /identity/v3/projects/76d247588aec429b89311037211e81d0/users/01b04eb348374397b58ec5cb16d64171/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 00:43:07.419799 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac0b4-7143-4807-b812-50ccb9b23935 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.420306 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac0b4-7143-4807-b812-50ccb9b23935 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.420584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac0b4-7143-4807-b812-50ccb9b23935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.451864 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-660ac0b4-7143-4807-b812-50ccb9b23935 None None] MFA Rules not processed for user `01b04eb348374397b58ec5cb16d64171`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:07.453206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-660ac0b4-7143-4807-b812-50ccb9b23935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:07.534754 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1429/2832] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:43:07.546680 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:07.548779 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:07.646282 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ba7d4-7274-48f8-980f-5c7cedbc0911 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:07.673799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:07.674767 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.675253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.675687 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.687745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:07.689693 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:07.726443 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1404/2833] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:07.776693 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05e3-7399-4cea-9f95-e2c2894bd8da None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:07.777254 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05e3-7399-4cea-9f95-e2c2894bd8da None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:07.777360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05e3-7399-4cea-9f95-e2c2894bd8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:07.808033 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a67d05e3-7399-4cea-9f95-e2c2894bd8da None None] MFA Rules not processed for user `7b59f1c6764b48579cc2f3039cd7c08b`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:07.809279 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a67d05e3-7399-4cea-9f95-e2c2894bd8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:07.940968 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1430/2834] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:09.983159 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:09.984579 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:09.999169 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:10.056720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edaef-2a43-4479-9b7d-f85678f2d034 service cinder] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:10.061939 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:10.062843 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.063121 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.063393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] PATH_INFO: `/v3/users/1a0ae44751124e81b32abee9af7494c5` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.064604 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0ae44751124e81b32abee9af7494c5)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:10.066511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:10.118628 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0ae44751124e81b32abee9af7494c5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.119833 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0ae44751124e81b32abee9af7494c5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.120446 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0ae44751124e81b32abee9af7494c5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.129250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0ae44751124e81b32abee9af7494c5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.137989 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0ae44751124e81b32abee9af7494c5'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.156134 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1405/2835] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:09 2022] DELETE /identity/v3/users/1a0ae44751124e81b32abee9af7494c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:10.160630 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.161129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.161247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.180283 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:10.181484 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:10.223862 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:10.244875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1431/2836] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:10.254478 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:10.255986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:10.316204 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2fc53-c570-4b6e-9099-9d8a9aaa6b18 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:10.320787 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:10.321440 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.321715 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.321986 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.332608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:10.334087 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344fa25-54e9-4ad3-b63d-a3f35ad7ae03 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:10.344845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1406/2837] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:10.624899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:10.627800 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:10.642649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73067d73-c9f2-4f95-96b4-a46c981cabed None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:10.650077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:10.651844 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.652124 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.652414 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] PATH_INFO: `/v3/projects/a057a550c0fb442ebaa9deb5ced441ba` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.653675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a057a550c0fb442ebaa9deb5ced441ba)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:10.655623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:10.716172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a057a550c0fb442ebaa9deb5ced441ba', 'request_id': 'req-2f542886-80fc-4432-872e-eb405e39a3d0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.722266 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a057a550c0fb442ebaa9deb5ced441ba', 'request_id': 'req-2f542886-80fc-4432-872e-eb405e39a3d0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.733443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a057a550c0fb442ebaa9deb5ced441ba', 'request_id': 'req-2f542886-80fc-4432-872e-eb405e39a3d0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.743567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] The token cache is being invalidate because project a057a550c0fb442ebaa9deb5ced441ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:10.743795 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.744496 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2f542886-80fc-4432-872e-eb405e39a3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:10.746041 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1432/2838] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:10 2022] DELETE /identity/v3/projects/a057a550c0fb442ebaa9deb5ced441ba => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:10.751677 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.751968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.752271 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.772186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:10.773393 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:10.806836 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:10.839162 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1407/2839] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:43:10.847281 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.847573 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.847858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.866104 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:10.867524 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:10.878868 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1433/2840] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:10.883242 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:10.893727 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5378e-8432-4940-ae63-f613af81ab67 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:10.898519 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:10.899225 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.899511 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.899896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] PATH_INFO: `/v3/domains` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.900683 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80266) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:43:10.900985 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:10.902590 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:10.914643 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1408/2841] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:43:10.919565 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:10.932980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85e5e-3717-4546-b890-bc1c28d8537f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:10.939409 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:10.940514 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:10.940976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:10.941490 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:10.942298 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:10.944242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:10.972649 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1434/2842] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:43:10 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:10.976877 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:10.988571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f46ab9-30d1-4c37-88a2-77cd9f95e53e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:11.047240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:11.047906 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.048182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.048452 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] PATH_INFO: `/v3/users` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.049160 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:11.051215 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:11.074106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1409/2843] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:43:10 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 00:43:11.077034 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:11.087362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b2ab7-c5ce-4e39-a1f6-b5c4e9682a16 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:11.095566 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:11.096670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.097129 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.097574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.099322 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:11.101569 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:11.115248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1435/2844] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:11.120164 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:11.130180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cb2d2-9802-45e1-b065-9761d7b6c193 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:11.135735 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:11.136424 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.136709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.136982 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.137709 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:11.139340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d303e6-787a-4743-b7d9-2bcfe7b6b25a admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:11.149538 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1410/2845] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:11.153875 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:11.164010 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239d35-e3da-4b2f-acc1-1cb50c849ef4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:11.168719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:11.170087 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.170458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.170857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] PATH_INFO: `/v3/projects/75e7882d623f4516aa4f73aa267e49fb/users/42f42f05c58344519c65ca4314d20d79/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.186493 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e7882d623f4516aa4f73aa267e49fb, user_id=42f42f05c58344519c65ca4314d20d79, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:11.189187 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:11.203962 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1436/2846] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:11 2022] PUT /identity/v3/projects/75e7882d623f4516aa4f73aa267e49fb/users/42f42f05c58344519c65ca4314d20d79/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:11.214631 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.214960 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.215238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.242144 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] MFA Rules not processed for user `42f42f05c58344519c65ca4314d20d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:11.243425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:11.308488 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1411/2847] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:11.321174 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:11.323623 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:11.396438 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8f98-f5d0-490e-a8cf-7a452452f3c6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:11.401278 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fb0a7dcacc745229341c1c02c575721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:11.402242 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:11.402699 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:11.403180 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:11.414081 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:11.416611 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c420242-e7dd-495b-a044-dcd51cc08e5d service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:11.428413 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1437/2848] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:17.151234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:17.188053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:17.219303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ff988-3dea-4c84-a5d2-e911da17ec4c None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:17.223904 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:17.224763 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:17.225040 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:17.225367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] PATH_INFO: `/v3/users/42f42f05c58344519c65ca4314d20d79` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:17.227114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f42f05c58344519c65ca4314d20d79)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:17.228990 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:17.276227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f42f05c58344519c65ca4314d20d79'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.276962 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f42f05c58344519c65ca4314d20d79'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.277571 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f42f05c58344519c65ca4314d20d79'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.285108 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f42f05c58344519c65ca4314d20d79'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.292760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f42f05c58344519c65ca4314d20d79'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.311368 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1412/2849] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:17 2022] DELETE /identity/v3/users/42f42f05c58344519c65ca4314d20d79 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:17.316830 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:17.317285 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:17.317723 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:17.332712 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:17.340273 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:17.342126 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:17.413881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1438/2850] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:17.424760 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:17.426269 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:17.486346 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4add6-1444-4279-a71e-911226fdf343 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:17.490071 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:17.490720 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:17.491031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:17.491308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:17.501852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:17.503429 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:17.513298 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1413/2851] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:17.792148 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:17.798405 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:17.819296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5afdd-ee5b-4aca-a8d0-77bf2b78acad None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:17.826756 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:17.827660 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:17.827943 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:17.828219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] PATH_INFO: `/v3/projects/75e7882d623f4516aa4f73aa267e49fb` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:17.829476 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e7882d623f4516aa4f73aa267e49fb)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:17.831458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:17.881635 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e7882d623f4516aa4f73aa267e49fb', 'request_id': 'req-9eb2d3f3-c258-4530-aa69-202be5033992'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.887500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e7882d623f4516aa4f73aa267e49fb', 'request_id': 'req-9eb2d3f3-c258-4530-aa69-202be5033992'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.894512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e7882d623f4516aa4f73aa267e49fb', 'request_id': 'req-9eb2d3f3-c258-4530-aa69-202be5033992'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.906485 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] The token cache is being invalidate because project 75e7882d623f4516aa4f73aa267e49fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:17.906770 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.907428 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9eb2d3f3-c258-4530-aa69-202be5033992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:17.909220 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1439/2852] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:17 2022] DELETE /identity/v3/projects/75e7882d623f4516aa4f73aa267e49fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:26.310100 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:26.311555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.324460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424bba6-2f3e-484e-b64c-43f7bb9c70e9 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:26.376292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:26.388923 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:26.390035 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:26.390471 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:26.391378 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] PATH_INFO: `/v3/users/01b04eb348374397b58ec5cb16d64171` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:26.392898 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b04eb348374397b58ec5cb16d64171)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:26.395649 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:26.444745 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b04eb348374397b58ec5cb16d64171'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:26.445736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b04eb348374397b58ec5cb16d64171'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:26.446558 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b04eb348374397b58ec5cb16d64171'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:26.455700 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b04eb348374397b58ec5cb16d64171'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:26.463234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b04eb348374397b58ec5cb16d64171'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:26.484869 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1414/2853] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:43:26 2022] DELETE /identity/v3/users/01b04eb348374397b58ec5cb16d64171 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:26.489309 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:26.489806 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:26.489895 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:26.509411 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:26.510721 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.587116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1440/2854] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:26.598365 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:26.600011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.662580 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f3b1-f4ee-4475-b167-2c10f0889fae admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:26.666250 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:26.666924 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:26.667865 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:26.668139 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:26.677295 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:26.677619 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:26.678708 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:26.679629 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.688340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1415/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:26.715708 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:26.755471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fe95c-5eba-47d7-8688-3618d4ae1016 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:26.762237 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:26.763232 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:26.763687 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:26.764134 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:26.767056 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.780391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:26.781997 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641162a3-7e4e-4a83-9e6b-65d5690f10c7 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:26.793342 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1441/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:43:26.968562 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:26.970366 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:26.985555 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee6015-31be-447f-b081-c18a623e4a66 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:26.991347 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:26.992200 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:26.992484 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:26.992759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] PATH_INFO: `/v3/projects/76d247588aec429b89311037211e81d0` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:26.994878 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d247588aec429b89311037211e81d0)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:26.996858 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.045342 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d247588aec429b89311037211e81d0', 'request_id': 'req-772db6ff-e410-4e58-a4ae-836ae73e125d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:27.050958 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d247588aec429b89311037211e81d0', 'request_id': 'req-772db6ff-e410-4e58-a4ae-836ae73e125d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:27.059231 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d247588aec429b89311037211e81d0', 'request_id': 'req-772db6ff-e410-4e58-a4ae-836ae73e125d'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:27.066917 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:27.072091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] The token cache is being invalidate because project 76d247588aec429b89311037211e81d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:43:27.072413 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:27.073216 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-772db6ff-e410-4e58-a4ae-836ae73e125d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:43:27.075739 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1416/2857] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:43:26 2022] DELETE /identity/v3/projects/76d247588aec429b89311037211e81d0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:27.081577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.082045 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.082502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.105865 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:27.108172 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:27.180443 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1442/2858] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:27.188234 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.188526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.188808 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.207363 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:27.208575 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:27.217106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1417/2859] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:27.221985 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.234007 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52abffc-acde-47dc-ac88-5469d9a39499 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.239976 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.240921 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.241373 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.241832 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.243518 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 00:43:27.244018 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.246262 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.260371 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1443/2860] 10.222.0.33 () {58 vars in 1152 bytes} [Wed Jun 15 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 00:43:27.265039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.274059 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23aa84-58a3-4813-ab07-d93b8db26daa None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.278737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.279912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.280188 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.280457 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.281350 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.283078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.308372 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1418/2861] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:43:27 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:27.313128 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.326378 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324087-6f09-40c7-b9da-d8b1c3156b42 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.402258 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.403584 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.404065 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.404760 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.406478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.409478 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.429763 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1444/2862] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:43:27 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:27.434736 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.445969 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11776780-84ca-4e87-b6d0-08789e556c90 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.451505 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.452171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.452455 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.452724 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.453459 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.455127 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.469406 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1419/2863] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:27.474319 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.484737 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f3d6b-272e-4614-bfd4-72dcbeee0458 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.499931 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.500918 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.501422 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.501801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.503097 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.504894 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c913f3-c143-4484-b7b2-27fceec6d5f7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.517299 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1445/2864] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:27.521305 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.533753 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cece-7fb2-41dd-a1be-d0a1b06a830b admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.539750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.540126 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.540227 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.540509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] PATH_INFO: `/v3/projects/2a609057ba0f4eb9aa2f90e7f31e0090/users/e6d41bf2ffcd487a93c05e51d673dfed/roles/b9e44997cbef45b98a11c68cadd4cbf3` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.558176 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a609057ba0f4eb9aa2f90e7f31e0090, user_id=e6d41bf2ffcd487a93c05e51d673dfed, role_id=b9e44997cbef45b98a11c68cadd4cbf3)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.561238 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.582001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1420/2865] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:27 2022] PUT /identity/v3/projects/2a609057ba0f4eb9aa2f90e7f31e0090/users/e6d41bf2ffcd487a93c05e51d673dfed/roles/b9e44997cbef45b98a11c68cadd4cbf3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:27.587200 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f27a-7b42-42d9-8b94-29e099ebabcf None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.587570 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f27a-7b42-42d9-8b94-29e099ebabcf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.587945 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f27a-7b42-42d9-8b94-29e099ebabcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.612646 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0ef0f27a-7b42-42d9-8b94-29e099ebabcf None None] MFA Rules not processed for user `ba726016075244549882bd6014777a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:27.613881 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ef0f27a-7b42-42d9-8b94-29e099ebabcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:27.637528 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:27.681857 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1446/2866] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 00:43:27.692394 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:27.694450 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:27.757665 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d4071-f19e-45dd-8cc0-b566d7f5fdeb admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:27.761106 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:27.761752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:27.762039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:27.762317 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:27.771789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:27.773171 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:27.782799 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1421/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:32.455672 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.456234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.456234 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.488677 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] MFA Rules not processed for user `e6d41bf2ffcd487a93c05e51d673dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:32.489934 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:32.581738 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1447/2868] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:32.585818 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.597032 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3336-ccd8-44bb-9918-81e788af135d service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.601688 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.602382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.602660 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.603043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] PATH_INFO: `/v3/projects` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.603759 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.605879 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.634925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1422/2869] 10.222.0.33 () {60 vars in 1152 bytes} [Wed Jun 15 00:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:32.639351 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.650446 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e9833-db3e-4159-aa5d-767a3e249a79 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.713568 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.714284 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.714563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.714866 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.715579 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.717498 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.739372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1448/2870] 10.222.0.33 () {60 vars in 1143 bytes} [Wed Jun 15 00:43:32 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 00:43:32.743387 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.754743 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454dc5-4845-4cdf-b756-60f1c5c92dff admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.759644 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.760308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.761156 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.761437 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.762173 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.763875 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.775292 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1423/2871] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:32.779667 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.790352 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cb1e3-bc90-4beb-b964-5ac6965113ee admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.795080 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.795969 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.796245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.796512 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] PATH_INFO: `/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d/users/ebcdf1a7cc9941acb880e68b7f08811c/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.813937 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c262ebf5f5e4e07b222c0dec1d4fa3d, user_id=ebcdf1a7cc9941acb880e68b7f08811c, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.816859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.831236 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1449/2872] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:32 2022] PUT /identity/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d/users/ebcdf1a7cc9941acb880e68b7f08811c/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:32.835148 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.845686 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e394-760f-4483-a723-6f10c2308285 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.904849 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.905642 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.905911 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.906182 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.906974 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.908601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a7286-cc15-4def-bf6e-dc04a0dd90e4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.914510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:32.920278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1424/2873] 10.222.0.33 () {58 vars in 1121 bytes} [Wed Jun 15 00:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 00:43:32.924058 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:32.935015 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75373-ce1f-43d9-b150-18ca2e183604 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:32.939750 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:32.940644 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:32.940925 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:32.941194 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] PATH_INFO: `/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d/users/ebcdf1a7cc9941acb880e68b7f08811c/roles/8d88da41d3264c6da23235143e5a2f57` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:32.943267 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c262ebf5f5e4e07b222c0dec1d4fa3d, user_id=ebcdf1a7cc9941acb880e68b7f08811c, role_id=8d88da41d3264c6da23235143e5a2f57)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:32.945913 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:32.957587 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1450/2874] 10.222.0.33 () {60 vars in 1482 bytes} [Wed Jun 15 00:43:32 2022] PUT /identity/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d/users/ebcdf1a7cc9941acb880e68b7f08811c/roles/8d88da41d3264c6da23235143e5a2f57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:43:37.312863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] REQUEST_METHOD: `POST` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:37.314153 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:37.314617 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:37.348258 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.auth.core [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] MFA Rules not processed for user `ebcdf1a7cc9941acb880e68b7f08811c`. Rule list: `[]` (Enabled: `True`). {{(pid=80266) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:37.349827 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:37.424449 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1425/2875] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:43:37.439025 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:37.440973 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:37.452905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:37.530727 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564ad17-df33-494b-bfc5-fbd4fab2c8bb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:37.536397 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:37.537046 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:37.537327 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:37.537603 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:37.596950 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:37.598860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defe881-2c0f-4889-a3bc-33db3007000e service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:37.609240 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1451/2876] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:41.196311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:43:41.198425 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:41.276091 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137436c4-78b7-4215-b980-c211d4463cb4 None None] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:43:41.280280 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:43:41.280953 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:41.281239 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:41.281635 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:41.284001 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:41.403006 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:43:41.404673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:43:41.426025 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1426/2877] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:43:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:43:47.604822 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=52 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:51.412367 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=84 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:43:52.989391 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:43:52.989915 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:43:52.990016 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:43:53.007666 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] MFA Rules not processed for user `e6d41bf2ffcd487a93c05e51d673dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:43:53.008858 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:43:53.016311 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1452/2878] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:43:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:44:03.002500 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:44:09.491154 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:44:09.500011 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a238f7-de3b-4c3e-b565-9dace64ac660 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:44:09.503608 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:44:09.504240 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:44:09.504522 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:44:09.504797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:44:09.513616 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:44:09.515075 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:44:09.524523 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1427/2879] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:44:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:44:19.520303 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:44:51.324387 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:44:51.334206 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85ea54-9ba7-474a-a8d5-5b0b1e3e1cab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:44:51.337746 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:44:51.338380 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:44:51.338662 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:44:51.338970 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:44:51.341265 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:44:51.350908 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:44:51.352330 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:44:51.360820 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1453/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:44:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:45:01.357270 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:45:40.752706 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:45:40.761235 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bae12-c595-450a-a359-81d2f27c8c65 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:45:40.764907 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:45:40.765567 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:45:40.765845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:45:40.766122 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:45:40.767929 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:45:40.781581 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:45:40.783031 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:45:40.794491 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1428/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:45:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:45:41.966561 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:45:41.975458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0364-0d7b-45e7-a7d9-a866b7cdce3f service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:45:41.979103 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:45:41.979732 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:45:41.980013 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:45:41.980294 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:45:41.982604 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:45:41.995817 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:45:41.997903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:45:42.009139 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1454/2882] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:45:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:45:50.789089 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:45:52.004127 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:45:55.697864 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:45:55.709186 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1fd7c-3029-4597-ab5a-7269471e0509 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:45:55.715284 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:45:55.715925 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:45:55.716206 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:45:55.716482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:45:55.718278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:45:55.731475 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:45:55.733078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:45:55.744679 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1429/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:45:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:45:56.377877 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:45:56.379676 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:45:56.448890 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447959-2742-48bd-a662-3fca9f0d468e service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:45:56.454605 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:45:56.455296 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:45:56.455551 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:45:56.456458 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:45:56.457654 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:45:56.470325 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:45:56.472221 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0509167-1933-4995-991c-7ffc1fb2e0ae service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:45:56.483159 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1455/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:45:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:46:05.740132 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:46:05.777012 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:46:05.789382 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a957e7-91dc-4448-aa78-3dcb7c7d31ab service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:46:05.800297 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:46:05.801065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:46:05.801613 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:46:05.802113 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:46:05.804737 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:46:05.819740 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:46:05.823334 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:46:05.842844 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1430/2885] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:46:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:46:06.479324 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=32 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:46:06.805899 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:46:06.806263 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:46:06.806607 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:46:06.825643 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] MFA Rules not processed for user `e6d41bf2ffcd487a93c05e51d673dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:46:06.826941 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:46:06.834274 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1456/2886] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:46:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:46:07.146592 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:46:07.155729 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afb8-ce17-4990-81c4-4ab30341afcd service nova] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:46:07.159264 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:46:07.159880 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:46:07.160155 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:46:07.160423 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:46:07.162189 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:46:07.171486 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:46:07.172891 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:46:07.180752 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1431/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:46:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:46:16.818651 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:46:17.177253 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=12 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:47:28.409966 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:28.412769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:28.464392 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1797-01fe-4307-842e-a65f591fcf63 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:28.471622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f57540b6cd47559628031b441a1eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:28.472275 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:28.472556 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:28.472828 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:28.478253 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:28.497808 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:28.499578 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:28.514027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1457/2888] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:47:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:47:35.272268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:35.273728 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:35.349293 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f4e78-4118-4255-bcb2-e6edd426aee4 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:35.355657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:35.356358 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:35.356647 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:35.356942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:35.358750 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:35.371509 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:35.373044 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:35.385466 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1432/2889] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:47:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:47:38.432079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:38.434574 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:38.545665 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e22f5-b39e-4cf0-9db3-412be4286401 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:38.550193 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:38.551077 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:38.551360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:38.551622 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] PATH_INFO: `/v3/users/7b59f1c6764b48579cc2f3039cd7c08b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:38.552801 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b59f1c6764b48579cc2f3039cd7c08b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:38.554702 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:38.600679 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b59f1c6764b48579cc2f3039cd7c08b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.601248 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b59f1c6764b48579cc2f3039cd7c08b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.608515 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b59f1c6764b48579cc2f3039cd7c08b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.617924 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b59f1c6764b48579cc2f3039cd7c08b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.630980 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b59f1c6764b48579cc2f3039cd7c08b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.637796 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1458/2890] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:47:38 2022] DELETE /identity/v3/users/7b59f1c6764b48579cc2f3039cd7c08b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:47:38.806104 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:38.822601 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef1586-5fdb-4449-942b-511aea27d8ab service placement] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:38.881328 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:38.882214 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:38.882500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:38.882791 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] PATH_INFO: `/v3/projects/5f0b03329de24df2b88d07327076ee88` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:38.883980 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0b03329de24df2b88d07327076ee88)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:38.885895 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:38.930931 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0b03329de24df2b88d07327076ee88', 'request_id': 'req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.936852 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0b03329de24df2b88d07327076ee88', 'request_id': 'req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.943813 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0b03329de24df2b88d07327076ee88', 'request_id': 'req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.954589 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] The token cache is being invalidate because project 5f0b03329de24df2b88d07327076ee88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:47:38.954908 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.955539 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:38.957065 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1433/2891] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:47:38 2022] DELETE /identity/v3/projects/5f0b03329de24df2b88d07327076ee88 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:47:38.961061 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:38.962471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:38.971747 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:47:38.975919 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81332a-0411-4f54-b245-4aa77180d8d4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:39.032502 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:39.033360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:39.033637 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:39.033907 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] PATH_INFO: `/v3/users/e1362472741448d5b93dc1c4b816fc39` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:39.035098 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1362472741448d5b93dc1c4b816fc39)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:39.037362 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:39.082640 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1362472741448d5b93dc1c4b816fc39'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.083307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1362472741448d5b93dc1c4b816fc39'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.090171 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1362472741448d5b93dc1c4b816fc39'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.097116 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1362472741448d5b93dc1c4b816fc39'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.109577 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3623910c-2adb-455e-8603-afae2e9a2d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1362472741448d5b93dc1c4b816fc39'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.115419 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1459/2892] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:47:38 2022] DELETE /identity/v3/users/e1362472741448d5b93dc1c4b816fc39 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:47:39.287968 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:47:39.289497 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:47:39.302845 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ca1a-030d-40f2-9e89-f3bf8ae93c8e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:47:39.344510 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:47:39.363136 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:47:39.363999 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:47:39.364278 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:47:39.364549 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] PATH_INFO: `/v3/projects/717858089df841db9420c6ce17671580` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:47:39.365721 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=717858089df841db9420c6ce17671580)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:47:39.367673 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:47:39.412319 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717858089df841db9420c6ce17671580', 'request_id': 'req-3beb9255-5af8-47cb-9e12-af3b219f9f9e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.417672 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717858089df841db9420c6ce17671580', 'request_id': 'req-3beb9255-5af8-47cb-9e12-af3b219f9f9e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.425371 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717858089df841db9420c6ce17671580', 'request_id': 'req-3beb9255-5af8-47cb-9e12-af3b219f9f9e'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.436213 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] The token cache is being invalidate because project 717858089df841db9420c6ce17671580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80266) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:47:39.436499 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.437120 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:47:39.438626 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1434/2893] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:47:39 2022] DELETE /identity/v3/projects/717858089df841db9420c6ce17671580 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:47:49.111961 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:47:49.433341 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:48:07.264670 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:48:07.265190 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:48:07.265307 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:48:07.282451 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] MFA Rules not processed for user `e6d41bf2ffcd487a93c05e51d673dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 00:48:07.283664 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:48:07.337131 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1460/2894] 10.222.0.33 () {58 vars in 957 bytes} [Wed Jun 15 00:48:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 00:48:07.645786 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:48:07.647302 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:48:07.725885 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb9255-5af8-47cb-9e12-af3b219f9f9e admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:48:07.729467 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:48:07.730103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:48:07.730383 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:48:07.730657 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:48:07.732441 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:48:07.741789 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:48:07.743257 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:48:07.752524 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1435/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:48:17.330586 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=25 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:48:17.748526 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=35 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:48:57.618567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:48:57.620072 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:48:57.713818 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bfa63-64e9-470b-94d6-7ed51fe22201 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:48:57.719563 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:48:57.720247 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:48:57.720534 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:48:57.720824 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:48:57.722602 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:48:57.735089 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:48:57.736700 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:48:57.746689 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1461/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:48:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:49:07.742360 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=41 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:49:56.334078 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:49:56.342882 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af071-1b0a-4a5c-b3f0-82c33fc219e9 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:49:56.346603 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:49:56.347268 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:49:56.347551 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:49:56.347825 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:49:56.349711 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:49:56.360482 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:49:56.361896 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:49:56.369916 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1436/2897] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:49:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:50:03.718535 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:03.728768 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8227dec-804e-4c92-a6ae-69c0e082d096 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:03.732526 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:03.733166 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:03.733447 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:03.733724 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:03.736138 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:03.746769 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:03.748216 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:03.756948 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1462/2898] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:50:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:50:06.366281 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=6 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:50:13.753079 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:50:17.060181 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:17.069335 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac574f4-1c5a-4f9c-8b1f-03bd4f628e7e service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:17.073283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd52fd2b7054851b62ae5b1d4f5ef29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:17.073909 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:17.074187 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:17.074460 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:17.076283 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:17.137930 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:17.139384 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:17.149797 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1437/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:50:20.170224 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:20.183460 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d464e-4ff8-4cee-932e-590e37924a49 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:20.188967 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:20.189618 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:20.189903 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:20.190176 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:20.192697 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:20.211859 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:20.213475 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:20.225471 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1463/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:50:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 00:50:23.262103 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:23.263824 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:23.276582 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd038-ea4f-42d1-8049-ff0bc532c289 service neutron] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:23.283500 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:23.284410 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:23.284696 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:23.285098 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] PATH_INFO: `/v3/users/e6d41bf2ffcd487a93c05e51d673dfed` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:23.286376 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d41bf2ffcd487a93c05e51d673dfed)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:23.288354 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:23.344652 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d41bf2ffcd487a93c05e51d673dfed'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.345252 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d41bf2ffcd487a93c05e51d673dfed'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.345914 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d41bf2ffcd487a93c05e51d673dfed'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.355311 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d41bf2ffcd487a93c05e51d673dfed'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.363180 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d41bf2ffcd487a93c05e51d673dfed'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.381742 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1438/2901] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:50:23 2022] DELETE /identity/v3/users/e6d41bf2ffcd487a93c05e51d673dfed => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:50:23.555334 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:23.566390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667a8e1-5952-4a0c-99b5-aa9a4e5b785e service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:23.626160 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:23.626963 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:23.627245 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:23.627516 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] PATH_INFO: `/v3/projects/2a609057ba0f4eb9aa2f90e7f31e0090` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:23.628675 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a609057ba0f4eb9aa2f90e7f31e0090)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:23.630567 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:23.674905 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a609057ba0f4eb9aa2f90e7f31e0090', 'request_id': 'req-3d2f971b-12d8-46a0-921f-efa79345720c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.680398 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a609057ba0f4eb9aa2f90e7f31e0090', 'request_id': 'req-3d2f971b-12d8-46a0-921f-efa79345720c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.687965 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a609057ba0f4eb9aa2f90e7f31e0090', 'request_id': 'req-3d2f971b-12d8-46a0-921f-efa79345720c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.698979 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] The token cache is being invalidate because project 2a609057ba0f4eb9aa2f90e7f31e0090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:50:23.699264 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.699893 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.701415 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1464/2902] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:50:23 2022] DELETE /identity/v3/projects/2a609057ba0f4eb9aa2f90e7f31e0090 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:50:23.706053 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:23.707504 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:23.720324 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fe1e-576c-44fc-a309-9b824e563da3 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:23.778175 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:23.779070 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:23.779534 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:23.779695 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] PATH_INFO: `/v3/users/ebcdf1a7cc9941acb880e68b7f08811c` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:23.780863 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcdf1a7cc9941acb880e68b7f08811c)` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:23.782764 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:23.788043 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:50:23.834195 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcdf1a7cc9941acb880e68b7f08811c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.834942 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcdf1a7cc9941acb880e68b7f08811c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.835546 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcdf1a7cc9941acb880e68b7f08811c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.843308 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcdf1a7cc9941acb880e68b7f08811c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.851340 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.notifications [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcdf1a7cc9941acb880e68b7f08811c'} {{(pid=80266) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:23.869718 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1439/2903] 10.222.0.33 () {58 vars in 1223 bytes} [Wed Jun 15 00:50:23 2022] DELETE /identity/v3/users/ebcdf1a7cc9941acb880e68b7f08811c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:50:24.045840 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:50:24.047432 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:50:24.064427 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f971b-12d8-46a0-921f-efa79345720c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:50:24.121581 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba726016075244549882bd6014777a2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc3f10f7de5140a991c9148b14dd9e73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:50:24.122437 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:50:24.122719 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:50:24.123027 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] PATH_INFO: `/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:50:24.124219 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c262ebf5f5e4e07b222c0dec1d4fa3d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:50:24.126860 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:50:24.147372 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:50:24.173469 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c262ebf5f5e4e07b222c0dec1d4fa3d', 'request_id': 'req-00962ca1-2437-4c36-a58c-b6b2940b0ee8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:24.179052 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c262ebf5f5e4e07b222c0dec1d4fa3d', 'request_id': 'req-00962ca1-2437-4c36-a58c-b6b2940b0ee8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:24.186678 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c262ebf5f5e4e07b222c0dec1d4fa3d', 'request_id': 'req-00962ca1-2437-4c36-a58c-b6b2940b0ee8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:24.198100 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] The token cache is being invalidate because project 7c262ebf5f5e4e07b222c0dec1d4fa3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 00:50:24.198390 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:24.199047 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00962ca1-2437-4c36-a58c-b6b2940b0ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 00:50:24.200583 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1465/2904] 10.222.0.33 () {58 vars in 1232 bytes} [Wed Jun 15 00:50:24 2022] DELETE /identity/v3/projects/7c262ebf5f5e4e07b222c0dec1d4fa3d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 00:50:33.866114 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:50:34.195177 dv-cinder-835384-6-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 00:54:03.598734 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Authenticating user token {{(pid=80266) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 00:54:03.600174 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:54:03.668313 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daa340-4ad5-4798-a60c-327580de48f4 admin admin] Validating token access rules against request {{(pid=80266) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 00:54:03.726820 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07f71bb8eebf4ec4999192359dafb28d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '491c878c8c7a4e489d2c511d19eb11e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80266) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 00:54:03.727758 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] REQUEST_METHOD: `GET` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 00:54:03.728039 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] SCRIPT_NAME: `/identity` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 00:54:03.728315 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80266) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 00:54:03.730101 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.fernet_utils [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80266) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 00:54:03.741912 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 00:54:03.743662 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e30f8-e767-4e0d-ae0f-05f64e446ae3 service placement] RBAC: Authorization granted {{(pid=80266) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 00:54:03.754508 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: [pid: 80266|app: 0|req: 1440/2905] 10.222.0.33 () {60 vars in 1365 bytes} [Wed Jun 15 00:54:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 00:54:13.749656 dv-cinder-835384-6-iscsi devstack@keystone.service[80266]: DEBUG dbcounter [-] [80266] Writing DB stats keystone:SELECT=58 {{(pid=80266) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}